Brain-inspired multimodal hybrid neural network for robot place recognition

计算机科学 机器人 人工智能 人工神经网络 人机交互
作者
Fangwen Yu,Yujie Wu,Songchen Ma,Mingkun Xu,Hongyi Li,Huanyu Qu,Chenhang Song,Taoyi Wang,Rong Zhao,Luping Shi
出处
期刊:Science robotics [American Association for the Advancement of Science (AAAS)]
卷期号:8 (78) 被引量:32
标识
DOI:10.1126/scirobotics.abm6996
摘要

Place recognition is an essential spatial intelligence capability for robots to understand and navigate the world. However, recognizing places in natural environments remains a challenging task for robots because of resource limitations and changing environments. In contrast, humans and animals can robustly and efficiently recognize hundreds of thousands of places in different conditions. Here, we report a brain-inspired general place recognition system, dubbed NeuroGPR, that enables robots to recognize places by mimicking the neural mechanism of multimodal sensing, encoding, and computing through a continuum of space and time. Our system consists of a multimodal hybrid neural network (MHNN) that encodes and integrates multimodal cues from both conventional and neuromorphic sensors. Specifically, to encode different sensory cues, we built various neural networks of spatial view cells, place cells, head direction cells, and time cells. To integrate these cues, we designed a multiscale liquid state machine that can process and fuse multimodal information effectively and asynchronously using diverse neuronal dynamics and bioinspired inhibitory circuits. We deployed the MHNN on Tianjic, a hybrid neuromorphic chip, and integrated it into a quadruped robot. Our results show that NeuroGPR achieves better performance compared with conventional and existing biologically inspired approaches, exhibiting robustness to diverse environmental uncertainty, including perceptual aliasing, motion blur, light, or weather changes. Running NeuroGPR as an overall multi–neural network workload on Tianjic showcases its advantages with 10.5 times lower latency and 43.6% lower power consumption than the commonly used mobile robot processor Jetson Xavier NX.
最长约 10秒,即可获得该文献文件

科研通智能强力驱动
Strongly Powered by AbleSci AI
科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
刚刚
哇咔咔完成签到,获得积分10
刚刚
Tink完成签到,获得积分10
刚刚
1秒前
1秒前
文静三颜发布了新的文献求助10
2秒前
小岛发布了新的文献求助30
2秒前
Doublelin发布了新的文献求助10
3秒前
兽行灵者发布了新的文献求助10
3秒前
平常幼菱完成签到,获得积分10
4秒前
4秒前
momo完成签到 ,获得积分10
6秒前
6秒前
原子发布了新的文献求助10
6秒前
爱听歌澜完成签到,获得积分10
6秒前
爱教育的张月亮完成签到,获得积分10
7秒前
momo发布了新的文献求助10
9秒前
兽行灵者完成签到,获得积分20
9秒前
10秒前
善学以致用应助sunwb采纳,获得10
10秒前
流浪的鲨鱼完成签到,获得积分20
10秒前
天66发布了新的文献求助10
11秒前
潦草完成签到,获得积分20
13秒前
13秒前
学学术术小小白白完成签到,获得积分10
13秒前
14秒前
沙不凡完成签到,获得积分10
14秒前
Active完成签到,获得积分10
15秒前
DAYTOY完成签到,获得积分10
16秒前
16秒前
16秒前
17秒前
Cynthia发布了新的文献求助10
17秒前
科目三应助witting采纳,获得10
19秒前
89238190发布了新的文献求助10
19秒前
20秒前
20秒前
21秒前
时倾发布了新的文献求助10
21秒前
深情安青应助傻瓜子采纳,获得10
22秒前
高分求助中
Production Logging: Theoretical and Interpretive Elements 2500
Востребованный временем 2500
Aspects of Babylonian celestial divination : the lunar eclipse tablets of enuma anu enlil 1500
Agaricales of New Zealand 1: Pluteaceae - Entolomataceae 1040
Healthcare Finance: Modern Financial Analysis for Accelerating Biomedical Innovation 1000
Classics in Total Synthesis IV: New Targets, Strategies, Methods 1000
지식생태학: 생태학, 죽은 지식을 깨우다 600
热门求助领域 (近24小时)
化学 医学 材料科学 生物 工程类 有机化学 生物化学 纳米技术 内科学 物理 化学工程 计算机科学 复合材料 基因 遗传学 物理化学 催化作用 细胞生物学 免疫学 电极
热门帖子
关注 科研通微信公众号,转发送积分 3459163
求助须知:如何正确求助?哪些是违规求助? 3053710
关于积分的说明 9037991
捐赠科研通 2742977
什么是DOI,文献DOI怎么找? 1504606
科研通“疑难数据库(出版商)”最低求助积分说明 695334
邀请新用户注册赠送积分活动 694663