MAE-EEG-Transformer: A transformer-based approach combining masked autoencoder and cross-individual data augmentation pre-training for EEG classification

Convolutional neural networks (CNN) may not be ideal for extracting global temporal features from non-stationary Electroencephalogram (EEG) signals. The application of the masking-based method in EEG classification is not well studied, and there is a shortage of commonly accepted models for verifying inter-individual results in motor imagery classification tasks. The MAE-EEG-Transformer, a transformer with masking mechanism, is proposed in this article. It pre-trains by randomly masking signals and forces the model to learn semantic features. The pre-trained encoder module is fine-tuned and moved to the classification task to obtain the category of EEG signals. The effectiveness of features with and without pre-training is compared using t-SNE visualization to demonstrate pre-training's inter-subject efficacy. The MAE EEG Transformer was extensively evaluated across three prevalent datasets in EEG-based motor imagery, demonstrating performance comparable to state-of-the-art models while requiring only approximately 20% of the computational cost (results in Table 1, 2, 3 and 4).