Interference-Robust Millimeter-Wave Radar-Based Dynamic Hand Gesture Recognition Using 2-D CNN-Transformer Networks

Dynamic gesture recognition using millimeter-wave radar has a broad application prospect in the industrial Internet of Things (IoT) field. However, the existing methods in the random dynamic interference environment, such as throwing objects and waving, easily cause wrong recognition. This paper proposes a dynamic gesture recognition method based on a CNNTransformer network to solve this problem. Firstly, we reshape the original echoes acquired by the frequency-modulated continuous-wave (FMCW) millimeter-wave radar into 3D data blocks in terms of Chirps × Samples × Frames. And we employ the mean elimination method to eliminate the static interference. Secondly, we extract dynamic gestures’ distance and Doppler information with the two-dimensional Fast Fourier Transform and obtain the range-time map and Doppler-time maps. And we employ the coherent accumulation method to improve the signalto-noise ratio (SNR). Thirdly, we construct the CNNTransformer network model for dynamic gesture recognition. The CNN is used to extract the local features of gestures, and multiple Transformer modules are stacked to extract deeper effective features. Finally, we build a data set for gesture recognition, including six kinds of dynamic gestures and two kinds of random interference. The experimental results show that the proposed method has a gesture recognition accuracy of more than 98% and 96% in the non-interference scene and the random dynamic interference scene, respectively, which are superior to the conventional recognition methods.