Hand gesture recognition based on range Doppler-angle trajectory and LSTM network using an MIMO radar

Touchless hand gesture recognition is of great importance for human-computer interaction (HCI). In this paper, we present a hand gesture recognition approach based on range-Doppler-angle trajectory and the long short-term memory (LSTM) network with a 77GHz frequency modulated continuous wave (FMCW) multiple-input-multiple-output (MIMO) radar. Firstly, the hand gesture fast-time-slow-time-antenna 3 dimension (3D) data are collected by the FMCW MIMO radar. Additionally, by performing the discretize Fourier transform (DFT) to the fast-time and slow-time, respectively, we obtain the range-profile and Doppler-profile. Then, by using the multiple signal classification (MUSIC) approach, we estimate the angle-profile of the hand gestures. To smooth and eliminate the noise effects, we apply the Kalman filtering to the estimated range-profile, Doppler-profile and angle-profile, respectively, and obtain the range-Doppler-angle trajectory signature. After that, by exploiting the temporal and spatial correlations, we construct a LSTM network for the hand gesture recognition. Experiments with 6 hand gestures are conducted and show that the proposed approach can recognize 6 hand gestures with an average accuracy over 97%.