Hear-Through and Mic-Through Augmented Reality: Using Bone Conduction to Display Spatialized Audio

We present a novel approach for mixing real and computer-generated audio for augmented reality (AR) applications. Analogous to optical-see-through and video-see-through techniques in the visual domain, we present Hear-Through and Mic-Through audio AR. Hear-Through AR uses a bone-conduction headset to deliver computer-generated audio, while leaving the ear canals free to receive audio from the surrounding environment. Mic-Through AR allows audio signals captured from ear-worn microphones to be mixed with computer-generated audio in the computer, and delivered to the user over headphones. We present preliminary results from an empirical user study conducted to compare a bone-conduction device, headphones, and a speaker array. The results show that subjects achieved the best accuracy using an array of speakers physically located around the listener when stationary sounds were played, but that there was no difference in accuracy between the speaker array and the bone-conduction device for sounds that were moving, and that both devices outperformed standard headphones for moving sounds.