MMHTSR: In-Air Handwriting Trajectory Sensing and Reconstruction Based on mmWave Radar

In-air Handwriting necessitates consistent motion tracking, in contrast to millimeter-wave (mmWave) radar-based simple gesture recognition techniques. However, during long-duration gesture tracking, challenges such as body motion interference and environmental clutter become more pressing. Moreover, due to the lack of a supporting surface in in-air handwriting, slight arm tremors also can result in unsmooth trajectories. To address these challenges, this paper proposes a two-stage processing framework called MMHTSR. In the first stage, the state-space equations are reestablished, and a locally correlated two-dimensional Gaussian process regression algorithm is employed for inter-frame prediction. By incorporating uncertainty estimation, weights are assigned to the next frame data, effectively suppressing interference from non-gestural targets. In the second stage, real-time smoothing and tracking of gesture trajectories are accomplished using a Kalman filter, followed by mapping the trajectories onto the Cartesian coordinate system. Finally, an end-to-end signal processing framework is deployed on a low-cost 60GHz mmWave radar prototype, and gesture trajectory recognition is achieved using deep learning methods. Experimental results demonstrate that MMHTSR can accurately track motion gestures within the range of approximately 5cm~40cm and successfully recognize 30 classes of in-air gesture trajectories, including uppercase letters A-Z and four interactive gesture actions. Furthermore, the proposed framework exhibits robust performance across various scenarios which shows its adaptability.