MIGO-NAS: Towards Fast and Generalizable Neural Architecture Search

计算机科学 水准点(测量) 人工神经网络 概化理论 人工智能 目标检测 管道(软件) 机器学习 分割 计算机工程 统计 数学 大地测量学 程序设计语言 地理
作者
Xiawu Zheng,Rongrong Ji,Yuhang Chen,Qiang Wang,Baochang Zhang,Jie Chen,Qixiang Ye,Feiyue Huang,Yonghong Tian
出处
期刊:IEEE Transactions on Pattern Analysis and Machine Intelligence [IEEE Computer Society]
卷期号:43 (9): 2936-2952 被引量:36
标识
DOI:10.1109/tpami.2021.3065138
摘要

Neural architecture search (NAS) has achieved unprecedented performance in various computer vision tasks. However, most existing NAS methods are defected in search efficiency and model generalizability. In this paper, we propose a novel NAS framework, termed MIGO-NAS, with the aim to guarantee the efficiency and generalizability in arbitrary search spaces. On the one hand, we formulate the search space as a multivariate probabilistic distribution, which is then optimized by a novel multivariate information-geometric optimization (MIGO). By approximating the distribution with a sampling, training, and testing pipeline, MIGO guarantees the memory efficiency, training efficiency, and search flexibility. Besides, MIGO is the first time to decrease the estimation error of natural gradient in multivariate distribution. On the other hand, for a set of specific constraints, the neural architectures are generated by a novel dynamic programming network generation (DPNG), which significantly reduces the training cost under various hardware environments. Experiments validate the advantages of our approach over existing methods by establishing a superior accuracy and efficiency i.e., 2.39 test error on CIFAR-10 benchmark and 21.7 on ImageNet benchmark, with only 1.5 GPU hours and 96 GPU hours for searching, respectively. Besides, the searched architectures can be well generalize to computer vision tasks including object detection and semantic segmentation, i.e., 25×25× FLOPs compression, with 6.4 mAP gain over Pascal VOC dataset, and 29.9×29.9× FLOPs compression, with only 1.41 percent performance drop over Cityscapes dataset. The code is publicly available.
最长约 10秒,即可获得该文献文件

科研通智能强力驱动
Strongly Powered by AbleSci AI
更新
PDF的下载单位、IP信息已删除 (2025-6-4)

科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
脑洞疼应助555557采纳,获得10
1秒前
点到为止发布了新的文献求助10
1秒前
4秒前
啵噜噜噜啊完成签到,获得积分10
7秒前
OYY完成签到 ,获得积分10
7秒前
烂漫的断秋完成签到 ,获得积分10
8秒前
9秒前
9秒前
Lucas应助Livrik采纳,获得10
10秒前
cyn0762完成签到,获得积分10
10秒前
点到为止完成签到,获得积分10
11秒前
12秒前
Akim应助lll采纳,获得10
13秒前
高贵灵槐完成签到 ,获得积分10
14秒前
17秒前
wxh完成签到 ,获得积分20
17秒前
量子星尘发布了新的文献求助10
17秒前
liuzi发布了新的文献求助20
18秒前
雨过天晴完成签到,获得积分10
19秒前
花花完成签到 ,获得积分10
20秒前
21秒前
23秒前
汪汪发布了新的文献求助10
26秒前
26秒前
27秒前
30秒前
追寻筮关注了科研通微信公众号
30秒前
miaojuly发布了新的文献求助10
31秒前
充电宝应助汪汪采纳,获得10
32秒前
33秒前
涵哈哈哈哈哈完成签到 ,获得积分10
35秒前
37秒前
liuzi完成签到,获得积分10
37秒前
科目三应助油菜的星星采纳,获得10
39秒前
39秒前
39秒前
40秒前
40秒前
木心应助高院士采纳,获得60
41秒前
41秒前
高分求助中
A new approach to the extrapolation of accelerated life test data 1000
ACSM’s Guidelines for Exercise Testing and Prescription, 12th edition 500
‘Unruly’ Children: Historical Fieldnotes and Learning Morality in a Taiwan Village (New Departures in Anthropology) 400
Indomethacinのヒトにおける経皮吸収 400
Phylogenetic study of the order Polydesmida (Myriapoda: Diplopoda) 370
基于可调谐半导体激光吸收光谱技术泄漏气体检测系统的研究 350
Robot-supported joining of reinforcement textiles with one-sided sewing heads 320
热门求助领域 (近24小时)
化学 材料科学 医学 生物 工程类 有机化学 生物化学 物理 内科学 纳米技术 计算机科学 化学工程 复合材料 遗传学 基因 物理化学 催化作用 冶金 细胞生物学 免疫学
热门帖子
关注 科研通微信公众号,转发送积分 3988868
求助须知:如何正确求助?哪些是违规求助? 3531255
关于积分的说明 11253071
捐赠科研通 3269858
什么是DOI,文献DOI怎么找? 1804822
邀请新用户注册赠送积分活动 881994
科研通“疑难数据库(出版商)”最低求助积分说明 809035