Contrastive Learning of Subject-Invariant EEG Representations for Cross-Subject Emotion Recognition

EEG signals have been reported to be informative and reliable for emotion\nrecognition in recent years. However, the inter-subject variability of\nemotion-related EEG signals still poses a great challenge for the practical\napplications of EEG-based emotion recognition. Inspired by recent neuroscience\nstudies on inter-subject correlation, we proposed a Contrastive Learning method\nfor Inter-Subject Alignment (CLISA) to tackle the cross-subject emotion\nrecognition problem. Contrastive learning was employed to minimize the\ninter-subject differences by maximizing the similarity in EEG signal\nrepresentations across subjects when they received the same emotional stimuli\nin contrast to different ones. Specifically, a convolutional neural network was\napplied to learn inter-subject aligned spatiotemporal representations from EEG\ntime series in contrastive learning. The aligned representations were\nsubsequently used to extract differential entropy features for emotion\nclassification. CLISA achieved state-of-the-art cross-subject emotion\nrecognition performance on our THU-EP dataset with 80 subjects and the publicly\navailable SEED dataset with 15 subjects. It could generalize to unseen subjects\nor unseen emotional stimuli in testing. Furthermore, the spatiotemporal\nrepresentations learned by CLISA could provide insights into the neural\nmechanisms of human emotion processing.\n