Classifying Individuals with ASD Through Facial Emotion Recognition and Eye-Tracking

Individuals with Autism Spectrum Disorder (ASD) have been shown to have atypical scanning patterns during face and emotion perception. While previous studies characterized ASD using eye-tracking data, this study examined whether the use of eye movements combined with task performance in facial emotion recognition could be helpful to identify individuals with ASD. We tested 23 subjects with ASD and 35 controls using a Dynamic Affect Recognition Evaluation (DARE) task that requires an individual to recognize one of six emotions (i.e., anger, disgust, fear, happiness, sadness, and surprise) while observing a slowly transitioning face video. We observed differences in response time and eye movements, but not in the recognition accuracy. Based on these observations, we proposed a machine learning method to distinguish between individuals with ASD and typically developing (TD) controls. The proposed method classifies eye fixations based on a comprehensive set of features that integrate task performance, gaze information, and face features extracted using a deep neural network. It achieved an 86% classification accuracy that is comparable with the standardized diagnostic scales, with advantages of efficiency and objectiveness. Feature visualization and interpretations were further carried out to reveal distinguishing features between the two subject groups and to understand the social and attentional deficits in ASD.