Facial Landmark Detection Based on High Precision Spatial Sampling via Millimeter-wave Radar

Facial landmark has become one of the most widely-used and intuitive feature of the face. Traditional methods for Facial Landmark Detection (FLD) are primarily based on cameras, which are limited by their sensitivity to light conditions, inability to penetrate obstructions, and potential privacy leaks. In this paper, we propose mmFLD to estimate the facial landmark positions using millimeter-wave (mmWave) radar with mm-level accuracy. By simultaneously applying the range estimation capability and angle estimation capability of mmWave radar, we are able to spatially sample face reflection signals with high resolution. In particular, we propose a velocity-based method for head detection and tracking, and then we build two generalized models to extract effective facial motion features from different facial regions. Moreover, we design an end-to-end neural network to extract the face structure and the motion coherence implicit in mmWave data. Experiment results show that mmFLD can estimate the facial landmark positions with high accuracy, e.g., the average Mean Absolute Error (MAE) is 2.81 mm with eight kinds of different facial expressions, and extended experiment also demonstrates the generalizability and robustness of mmFLD for different experiment conditions.