mDS-PCGR: A Bi-Modal Gait Recognition Framework With the Fusion of 4D Radar Point Cloud Sequences and micro-Doppler Signatures

Radar-based gait recognition has emerged as a promising solution for non-invasive human identification. However, relying solely on single-modal radar gait representations, such as micro-Doppler signature and radar point cloud, proves inadequate for robust gait recognition in the presence of complex perceptual conditions. Additionally, achieving a high level of generalization, particularly when dealing with new subjects having limited training samples, is crucial for practical gait recognition. To address these challenges, we present a novel joint micro-Doppler and radar point clouds gait recognition framework (mDS-PCGR) in this study. This framework fuses gait features derived from both micro-Doppler signatures and four-dimension (4D) radar point cloud sequences. Firstly, a tracking-based preprocessing method is proposed to acquire high-quality micro-Doppler signatures and 4D radar point cloud sequences, while suppressing the multipath interference in complex perceptual conditions. Secondly, a dual-flow fusion network is designed to extract discriminative gait features based on complementation of the two modalities to each other. Finally, a metric-based few-shot learning mechanism is used to instruct the optimization of dual-flow fusion network, combining triplet loss with center loss to achieve the identification of new subjects with few training samples. Extensive evaluation on real 4D millimeter-wave radar measurement under multipath interfered and cross-view conditions is provided. Experimental results show the superior performance of the proposed mDS-PCGR, leveraging effective gait information from two modalities. It outperforms single-modal gait recognition methods and achieves the highest identification accuracy for new subjects with limited gallery samples.