Investigating effects of multimodal explanations using multiple In-vehicle displays for takeover request in conditionally automated driving

In conditionally automated driving, a human takeover transition is necessary in response to vehicle malfunction or other limitation. To mitigate the negative experience of this technological limitation, researchers have suggested providing an explanation along with the takeover request to increase drivers' understanding and trust in an automated driving system (ADS). However, previous studies on the explanation have been limited to investigations of unimodal explanations that use either text or speech, while neglecting other various in-vehicle display options. The present study aims to investigate the effects of multimodal explanation using three in-vehicle displays—ambient light, cluster, and sound. The explanation given by each in-vehicle display was systemically manipulated into two levels (ambient light: constant color vs. variable color, cluster: no-text and no-background vs. text with colored background, sound: alert-only sound vs. speech with alert sound). To examine the effects of the explanation using these displays, we conducted mixed method research including a scenario-based experiment, questionnaires, and a follow-up interview for thematic analysis. In the experiment, eighty-two participants drove in a driving simulator and performed takeover tasks in several types of urgent situations for measuring their takeover time and error. The participants then responded to questionnaires on takeover workload and overall trust in automated vehicle. Next, fifty-six of them participated in the follow-up interview for discovering significant themes related to their experiences with the multimodal explanation. As results, we found that multimodal explanations that use two types of displays were more effective than unimodal explanations or multimodal explanations using three types of displays. Regarding the effects of each type of display, the speech-based explanation showed the best takeover performance in general. The explanation using the cluster was next most useful in performance. The ambient light explanation showed less significant to no effect in the takeover situation. We expect that these findings can contribute to theoretical understanding of driver response to multimodal explanations and to practical design of trustworthy explanations in various takeover situations.