A Novel Multimodal Human-Exoskeleton Interface Based on EEG and sEMG Activity for Rehabilitation Training

Despite the advances in the field of human-robot interface (HRI) based on biological neural signal, the use of the sole electroencephalography (EEG) signal to help robotic exoskeleton predict the limb movement is currently no mature in rehabilitation training, due to its unreliability. Multimodal HRI represents a very recent solution to enhance the performance of single modal HRI. These HRI normally include the EEG signal with surface electromyography (sEMG) signal. However, their use for the lower limb movement prediction in hemiplegia is still limited, and the deep fusion feature of sEMG and EEG signal is ignored. This paper proposes a Dense co-attention mechanism-based Multimodal Enhance fusion Network (DMEFNet) for the lower limb movement prediction in hemiplegia. The DMEFNet can realize the mapping and deep fusion between the sEMG and EEG signal features and get a high accuracy movement prediction of the lower limbs. A sEMG and EEG data acquisition experiment and an incomplete asynchronous data collection paradigm are designed to verify the effectiveness of DMEFNet. The experimental results show that DMEFNet has a good movement prediction performance in both within-subject and cross-subject situations, reaching an accuracy of 82.96% and 88.44% respectively.