Ultrasound-Based 3-D Gesture Recognition: Signal Optimization, Trajectory, and Feature Classification

Gesture recognition is an emerging technology of human–computer interaction. Compared to the conventional technologies such as wearable devices, cameras, and radars, ultrasound-based gesture recognition has the advantages of low cost, low power consumption, and appropriates to fog or dark environment. In this study, we propose a gesture trajectory recognition device via ultrasonic signal, which is successfully applied in recognition and classification for gestures in 3-D space. First, the sensing hardware and software configurations for transmitting and receiving ultrasonic signals are designed, which can accurately obtain ultrasonic echo. Effects of different structures on the ultrasound scattering are explored. An adaptive filter based on wavelet packet decomposition is utilized to remove the noise in the ultrasonic echo. Second, the vibration principle of the ultrasonic sensor is modeled, and the rising envelope of ultrasound is fit by the quadratic curve to accurately extract the time of flight (TOF). Then, the error analysis and compensation are performed. An active 3-D positioning model is constructed and the positioning and tracking algorithm of gesture motion is theoretically established. A binomial fitting algorithm-based sliding template is proposed. To process the abnormal value and locate the start position of the gesture, the data association criterion and the maximum displacement threshold are introduced. Projection, dimensionality reduction, and reconstruction are performed on the trajectory. Then, the distance feature and the direction feature are extracted, and the hidden Markov model (HMM) is applied to realize the classification and recognition of gesture trajectory. The experiment shows that the 3-D gesture recognition system based on ultrasound can recognize a total of 36 character gestures from 0 to 9 and A to Z, with a recognition rate of 90.53%.