Towards Reliable Multimodal Stress Detection under Distribution Shift

The recognition of stress is an important issue from a health care perspective as well as in the human-computer interaction context. With the help of multimodal sensors, stress can be detected relatively well under laboratory conditions. However, when models are used in the real world, shifts in the data distribution can occur, often leading to performance degradation. It is therefore desirable that models in these scenarios are at least able to accurately capture this uncertainty and thus know what they do not know. This work aims to investigate how synthetic shifts in the data distribution can affect the reliability of a multimodal stress detection model in terms of calibration and uncertainty quantification. We compare a baseline with three known approaches that aim to improve reliability of uncertainty estimates. Our results show that all methods we tested improve the calibration. However, calibration generally deteriorates and spreads with stronger shifts for all approaches. They perform especially poorly for shifts in highly relevant modalities. Overall, we conclude that in the conducted experiments the investigated methods are not sufficiently reliable under distribution shifts.