MSFNet: A Multi-Scale Space-Time Frequency Fusion Network for Motor Imagery EEG Classification

Motor imagery electroencephalogram (MI-EEG) classification is essential in brain-computer interface (BCI), and many classification methods have been proposed recently. However, the MI-EEG classification accuracy of the public dataset still has room for improvement, and designing a suitable model to extract and fuse the multi-modality features efficiently is crucial. In this study, we proposed a Multi-scale Space-time Frequency fusion Network (MSFNet) to improve the MI-EEG classification accuracy. The MSFNet comprises data acquisition and preprocessing, multi-scale time-conv fusion unit, multi-scale frequency-conv fusion unit, feature fusion, and classification. Multi-scale time-conv fusion unit can extract multi-scale spatiotemporal features, and multi-scale frequency-conv fusion unit can extract five frequency sub-band features. These two features were concatenated to complete the multi-modality features fusion, and MI-EEG was classified. Average accuracy, kappa value, and F1 score were adopted as the evaluation metric, and BCI Competition 2008 IV 2a and High Gamma datasets were employed to demonstrate the effectiveness of the MSFNet. The superiority of this proposed model was demonstrated by comparison against the state-of-the-art methods, and the classification result is the highest. Overall, we achieved an average accuracy of 80.47%, kappa value of 0.783, and F1 score of 0.743 in the BCI Competition 2008 IV 2a dataset, and we achieved an average accuracy of 93.56%, kappa value of 0.933, and F1 score of 0.915 in High Gamma dataset. This model realized the extraction and efficient fusion of spatiotemporal and frequency domain features and obtained the high-precision MI-EEG classification, which has an important application value.