Head-up Displays Improve Drivers’ Performance and Subjective Perceptions with the In-Vehicle Gesture Interaction System

In-vehicle infotainment systems can cause various distractions, increasing the risk of car accidents. To address this problem, mid-air gesture systems have been introduced. This study investigated the potential of a novel interface that integrates a Head-Up Display (HUD) with auditory displays (spearcons: compressed speech) in a gesture-based menu navigation system to minimize visual distraction and improve driving and secondary task performance. The experiment involved 24 participants who navigated through 12 menu items using mid-air gestures while driving on a simulated road under four conditions: HUD (with, without spearcons) and Head-Down Display (HDD) (with, without spearcons). Results showed that the HUD condition significantly outperformed the HDD condition in participants' level 1 situation awareness, perceived workload, menu navigation performance, and system usability. However, there were trade-offs on visual fixation duration on the menu, and lane deviation. These findings will guide future research in developing safer and more effective HUD-supported in-vehicle gesture interaction systems.