Region-Based Illumination-Temperature Awareness and Cross-Modality Enhancement for Multispectral Pedestrian Detection

多光谱图像 模态(人机交互) 行人检测 计算机视觉 遥感 行人 人工智能 计算机科学 环境科学 材料科学 地理 工程类 运输工程
作者
Yanhao Liu,Chuan Hu,Baixuan Zhao,Yonghui Huang,Xi Zhang
出处
期刊:IEEE transactions on intelligent vehicles [Institute of Electrical and Electronics Engineers]
卷期号:: 1-12 被引量:3
标识
DOI:10.1109/tiv.2024.3367688
摘要

Multispectral pedestrian detection based on RGB-thermal (RGB-T) camera has been actively studied in autonomous driving in recent years as its robustness under complex traffic scenes. However, the fusion of multispectral data poses several challenges. Firstly, the fusion method requires dynamic adjustment of fusion weights considering environmental influences, such as illumination and temperature. Secondly, effective feature fusion necessitates addressing slight misalignment of visual sensors and enhancement of inconspicuous target's feature in traffic scenes. To solve problems above, we propose a novel network with three effective modules. In contrast to previous global fusion weight methods, the region-based illumination and temperature aware (RITA) module is proposed as dual pipeline structure to generate 5 regional fusion weights, which contains global and regional environmental information comprehensively. Additionally, compared to previous one-stage fusion strategies, a two-stage refined modality fusion is proposed by two modules. The spatial-aligned modal fusion (SAMF) module generates fusion features with large-scale spatial attention masks, which can enhance corresponding features and alleviate the slight misalignment between different modalities. The object-correlated cross-modality enhancement (OCE) module is proposed to complement effective features to fusion modality, which establishes inter-pedestrian relationships and enhance features of inconspicuous pedestrians. Experimental results of average miss rate on two challenging multispectral pedestrian datasets KAIST and CVC-14 achieve 7.64% and 21.3% respectively, and outperform competitive BAANet by 10.35% in miss rate of distant pedestrians in KAIST, demonstrating the advantages of our proposed method compared with state-of-the-art methods.
最长约 10秒,即可获得该文献文件

科研通智能强力驱动
Strongly Powered by AbleSci AI
科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
刚刚
liubai发布了新的文献求助10
1秒前
在水一方应助重要听荷采纳,获得10
1秒前
梵梵完成签到 ,获得积分10
1秒前
吐司发布了新的文献求助10
1秒前
2秒前
liubai发布了新的文献求助50
2秒前
liubai发布了新的文献求助10
3秒前
liubai发布了新的文献求助30
3秒前
失眠的zth完成签到,获得积分10
5秒前
liubai发布了新的文献求助10
5秒前
liubai发布了新的文献求助10
5秒前
liubai发布了新的文献求助10
5秒前
liubai发布了新的文献求助10
5秒前
甜甜完成签到,获得积分20
6秒前
6秒前
Zhou_zp发布了新的文献求助10
7秒前
zzq完成签到,获得积分10
8秒前
超人爱吃菠菜完成签到,获得积分10
8秒前
yourself完成签到,获得积分10
11秒前
小二郎应助H的流年采纳,获得10
11秒前
斯文败类应助满当当采纳,获得10
12秒前
海韵_Tony发布了新的文献求助10
13秒前
lg20010419完成签到,获得积分10
14秒前
kelly完成签到,获得积分10
14秒前
Ava应助wenlong采纳,获得10
15秒前
maox1aoxin应助Hoshino采纳,获得50
17秒前
万能图书馆应助沐夏采纳,获得10
21秒前
Lw完成签到,获得积分10
21秒前
22秒前
23秒前
大模型应助海韵_Tony采纳,获得10
23秒前
23秒前
Zhou_zp完成签到,获得积分10
25秒前
zhuhuaipu完成签到 ,获得积分10
25秒前
万能图书馆应助阿拉采纳,获得10
25秒前
26秒前
LiYaru完成签到,获得积分10
28秒前
CipherSage应助小鸭飞采纳,获得10
29秒前
LiYaru发布了新的文献求助10
31秒前
高分求助中
(应助此贴封号)【重要!!请各用户(尤其是新用户)详细阅读】【科研通的精品贴汇总】 10000
Modern Epidemiology, Fourth Edition 5000
Kinesiophobia : a new view of chronic pain behavior 5000
Molecular Biology of Cancer: Mechanisms, Targets, and Therapeutics 3000
Digital Twins of Advanced Materials Processing 2000
Weaponeering, Fourth Edition – Two Volume SET 2000
Signals, Systems, and Signal Processing 610
热门求助领域 (近24小时)
化学 材料科学 医学 生物 工程类 有机化学 纳米技术 化学工程 生物化学 物理 计算机科学 内科学 复合材料 催化作用 物理化学 光电子学 电极 冶金 细胞生物学 基因
热门帖子
关注 科研通微信公众号,转发送积分 6018327
求助须知:如何正确求助?哪些是违规求助? 7606399
关于积分的说明 16158938
捐赠科研通 5165921
什么是DOI,文献DOI怎么找? 2765127
邀请新用户注册赠送积分活动 1746656
关于科研通互助平台的介绍 1635331