ADReFV : Face video dataset based on human‐computer interaction for Alzheimer's disease recognition

Abstract With the global aging problem becoming more and more serious, the initial screening for Alzheimer's disease (AD) will become increasingly important. We understand that facial expressions are related to the severity of dementia, but there is no face‐related data in the existing Alzheimer's dataset. This article attempts to establish a facial video‐based AD recognition dataset through a human‐computer interaction method. This interactive task was designed for AD in attention, execution, visual space ability, facial apraxia, and facial changes in task success and failure. Using this task as the collection method, the final dataset includes 102 faces video data, specific task scores, and emotional self‐evaluation. For baseline evaluation, the improved local binary pattern on three orthogonal planes and RF were employed respectively for feature extraction and classification with the 5‐fold cross‐validation method. The best performance was 76.00% for 3‐class classification. In addition, a frame attention network based on fine‐grained local region localization was proposed, which improved the accuracy of cognitive classification to 84.45%. Finally, the analysis was conducted for the association of expressions with cognition and emotion in the AD dataset. This study aims to solve the current lack of standards for AD in the field of facial recognition and contribute to future research and clinical applications.