Applying Sonification to Sketching in the Air With Mobile AR Devices

With more and more mobile devices (such as smart phones and tablets) supporting augmented reality (AR), using these devices to sketch in mid-air has become a popular application direction. However, due to the small display size of these devices, the user's field of view is limited and cannot see the context of graphics, which leads to the drawn graphics deviating from their expectations. In this article, we applied sonification technology to mid-air sketching with mobile AR device, and proposed a new method to address this problem. In our first experiment, we verified the feasibility of our method. Our experimental results showed that sonification can effectively reduce the deviation caused by a narrow field of view. In our second experiment, we further explored the application ability of this method in a wider range of sketching. Our experimental results showed that sonification can improve the aspect ratio of the drawn graphics. In addition, the results of the NASA-TLX questionnaire showed that the participants' mental demand and effort decreased significantly, and their subjective performance increased significantly. We proposed a new method, which can effectively improve the accuracy of mid-air sketching with a mobile AR device.