Fusion Analysis of EEG-fNIRS Multimodal Brain Signals: A Multitask Classification Algorithm Incorporating Spatial-Temporal Convolution and Dual Attention Mechanisms

脑电图 计算机科学 卷积(计算机科学) 人工智能 模式识别(心理学) 语音识别 功能近红外光谱 对偶(语法数字) 神经影像学 算法 人工神经网络 神经科学 心理学 认知 前额叶皮质 艺术 文学类
作者
Xingbin Shi,Haiyan Wang,Baojiang Li,Yuxin Qin,Cheng Peng,Yifan Lu
出处
期刊:IEEE Transactions on Instrumentation and Measurement [Institute of Electrical and Electronics Engineers]
卷期号:74: 1-12 被引量:7
标识
DOI:10.1109/tim.2025.3538086
摘要

Brain–computer interface (BCI) is an important way of human-computer interaction, with the ability to monitor brain states, and it has become an increasingly significant research direction. Single-modal noninvasive brain signals have limitations, such as low spatial resolution or low temporal resolution, while multimodal brain signal acquisition and processing can overcome these limitations. Electroencephalogram and functional near-infrared spectroscopy (EEG-fNIRS) is a method with advantages in multimodal brain signal processing, but current fusion methods mostly use manual feature extraction or channel selection, which may lead to the loss of important information during the feature extraction or channel selection process in real-time BCI systems. In order to solve this issue, this article proposes an innovative fusion analysis method for EEG-fNIRS multimodal brain signals, using a hybrid algorithm that combines convolutional neural network (CNN) and Attention mechanisms for signal classification. The method first preprocesses the EEG and fNIRS signals separately, then extracts features using spatial-temporal convolutional layers, and finally merges them to obtain the classification results through dual attention calculation. Our method is validated on two publicly available mixed EEG-fNIRS BCI datasets, including three types of experimental tasks that do not involve actual movement: motor imagery (MI), mental arithmetic, and word generation (WG). The accuracy rates for each task reached 92.2% for MI, 98.6% for mental arithmetic, and 95.2% for WG, respectively. These rates have surpassed all the current methods. This indicates that our proposed method achieves better classification performance in non-actual movement classification tasks under the premise of lightweight. The method proposed in this study can be applied to the field of rapid and efficient identification of brain signals.
最长约 10秒,即可获得该文献文件

科研通智能强力驱动
Strongly Powered by AbleSci AI
科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
刚刚
1秒前
星城浮轩完成签到 ,获得积分10
2秒前
342396102完成签到,获得积分10
2秒前
微笑萝完成签到 ,获得积分10
2秒前
幸运花花完成签到,获得积分10
3秒前
3秒前
wanyue完成签到,获得积分10
3秒前
Xiaoxin_Ju完成签到,获得积分10
3秒前
胡雅琴完成签到,获得积分10
3秒前
Tues发布了新的文献求助10
3秒前
Venus完成签到,获得积分0
3秒前
111完成签到,获得积分10
4秒前
沉静青寒完成签到,获得积分10
4秒前
喵喵666完成签到,获得积分10
4秒前
Aikesi完成签到,获得积分10
4秒前
lizishu应助Maud采纳,获得20
5秒前
5秒前
5秒前
务实幻露发布了新的文献求助10
6秒前
6秒前
hhhhhhhh完成签到,获得积分10
7秒前
7秒前
半岛完成签到,获得积分10
7秒前
星星点灯完成签到,获得积分10
7秒前
科研通AI6.2应助阿豆采纳,获得10
8秒前
兔兔完成签到,获得积分10
8秒前
8秒前
KOBE94FU完成签到,获得积分10
9秒前
9秒前
梁书凡发布了新的文献求助10
9秒前
王二哈完成签到,获得积分10
10秒前
10秒前
笑点低翠琴完成签到,获得积分10
10秒前
Zachary完成签到,获得积分10
10秒前
简历发布了新的文献求助10
10秒前
MZT完成签到,获得积分10
10秒前
热心的诗蕊完成签到,获得积分10
11秒前
榆树畔完成签到,获得积分10
11秒前
Woke发布了新的文献求助10
11秒前
高分求助中
(应助此贴封号)【重要!!请各用户(尤其是新用户)详细阅读】【科研通的精品贴汇总】 10000
Modern Epidemiology, Fourth Edition 5000
Kinesiophobia : a new view of chronic pain behavior 5000
Molecular Biology of Cancer: Mechanisms, Targets, and Therapeutics 3000
Digital Twins of Advanced Materials Processing 2000
Propeller Design 2000
Weaponeering, Fourth Edition – Two Volume SET 2000
热门求助领域 (近24小时)
化学 材料科学 医学 生物 工程类 有机化学 纳米技术 化学工程 生物化学 物理 计算机科学 内科学 复合材料 催化作用 物理化学 光电子学 电极 冶金 细胞生物学 基因
热门帖子
关注 科研通微信公众号,转发送积分 6013555
求助须知:如何正确求助?哪些是违规求助? 7583697
关于积分的说明 16141351
捐赠科研通 5160886
什么是DOI,文献DOI怎么找? 2763448
邀请新用户注册赠送积分活动 1743606
关于科研通互助平台的介绍 1634401