A tree-structure-guided graph convolutional network with contrastive learning for the assessment of parkinsonian hand movements

Bradykinesia is one of the core motor symptoms of Parkinson's disease (PD). Neurologists typically perform face-to-face bradykinesia assessment in PD patients according to the Movement Disorder Society-sponsored revision of the Unified Parkinson's Disease Rating Scale (MDS-UPDRS). As this human-expert assessment lacks objectivity and consistency, an automated and objective assessment scheme for bradykinesia is critically needed. In this paper, we propose a tree-structure-guided graph convolutional network with contrastive learning scheme to solve the challenge of difficulty in fine-grained feature extraction and insufficient model stability, finally achieving the video-based automated assessment of Parkinsonian hand movements, which represent a vital MDS-UPDRS component for examining upper-limb bradykinesia. Specifically, a tri-directional skeleton tree scheme is proposed to achieve effective fine-grained modeling of spatial hand dependencies. In this scheme, hand skeletons are extracted from videos, and then the spatial structures of these skeletons are constructed through depth-first tree traversal. Afterwards, a tree max-pooling module is employed to establish remote exchange between outer and inner nodes, hierarchically gather the most salient motion features, and hence achieve fine-grained mining. Finally, a group-sparsity-induced momentum contrast is also developed to learn similar motion patterns under different interference through contrastive learning. This can promote stable learning of discriminative spatial-temporal features with invariant motion semantics. Comprehensive experiments on a large clinical video dataset reveal that our method achieves competitive results, and outperforms other sensor-based and RGB-depth methods. The proposed method leads to accurate assessment of PD bradykinesia through videos collected by low-cost consumer cameras of limited capabilities. Hence, our work provides a convenient tool for PD telemedicine applications with modest hardware requirements.