Face2Nodes: Learning facial expression representations with relation-aware dynamic graph convolution networks

Deep convolutional neural networks (CNNs) have become the standard model architecture for facial expression recognition (FER). However, CNN-based models struggle to capture the structural correlations between different local regions in a face image. Recent methods based on Vision Transformer (ViT) have been introduced to capture long-range dependencies among local regions. Nonetheless, ViT-based approaches are vulnerable to facial regions unrelated to expressions and may learn redundant correlation representations due to their self-attention mechanism. To address these issues, we propose a novel graph-based model called Face2Nodes, which can flexibly learn the graph representations of facial expressions without requiring additional auxiliary facial information such as landmarks. Our Face2Nodes consists of two key components: a multi-scale feature fusion-based patch embedding and a relation-aware dynamic graph convolution network. The patch embedding method uses a multi-scale feature fusion mechanism to obtain more discriminative graph node features for further graph representation learning. A dynamic graph is constructed using the dilated k-nearest neighbors algorithm, and a relation-aware graph convolution operator is designed to learn the latent informative correlations among different nodes in the graph. Extensive experiment results show that Face2Nodes achieves state-of-the-art performance on several popular in-the-wild FER datasets, with overall accuracies of 91.41%, 91.02%, and 66.69% on the FERPlus, RAF-DB, and AffectNet databases, respectively. Furthermore, we found that CNN-based FER approaches have a more significant performance gap between pre-training and training from scratch than Face2Nodes, demonstrating that our model is more data-efficient than CNN-based approaches.