Universal and extensible language-vision models for organ segmentation and tumor detection from abdominal computed tomography

计算机科学 可扩展性 分割 人工智能 编码(内存) 语言模型 灵活性(工程) 方案(数学) 软件 体素 机器学习 自然语言处理 程序设计语言 数学分析 统计 数学
作者
Jie Liu,Yixiao Zhang,Kang Wang,Mehmet Can Yavuz,Xiaoxi Chen,Yixuan Yuan,Haoliang Li,Yang Yang,Alan Yuille,Yucheng Tang,Zongwei Zhou
出处
期刊:Medical Image Analysis [Elsevier BV]
卷期号:97: 103226-103226 被引量:35
标识
DOI:10.1016/j.media.2024.103226
摘要

The advancement of artificial intelligence (AI) for organ segmentation and tumor detection is propelled by the growing availability of computed tomography (CT) datasets with detailed, per-voxel annotations. However, these AI models often struggle with flexibility for partially annotated datasets and extensibility for new classes due to limitations in the one-hot encoding, architectural design, and learning scheme. To overcome these limitations, we propose a universal, extensible framework enabling a single model, termed Universal Model, to deal with multiple public datasets and adapt to new classes (e.g., organs/tumors). Firstly, we introduce a novel language-driven parameter generator that leverages language embeddings from large language models, enriching semantic encoding compared with one-hot encoding. Secondly, the conventional output layers are replaced with lightweight, class-specific heads, allowing Universal Model to simultaneously segment 25 organs and six types of tumors and ease the addition of new classes. We train our Universal Model on 3410 CT volumes assembled from 14 publicly available datasets and then test it on 6173 CT volumes from four external datasets. Universal Model achieves first place on six CT tasks in the Medical Segmentation Decathlon (MSD) public leaderboard and leading performance on the Beyond The Cranial Vault (BTCV) dataset. In summary, Universal Model exhibits remarkable computational efficiency (6× faster than other dataset-specific models), demonstrates strong generalization across different hospitals, transfers well to numerous downstream tasks, and more importantly, facilitates the extensibility to new classes while alleviating the catastrophic forgetting of previously learned classes. Codes, models, and datasets are available at https://github.com/ljwztc/CLIP-Driven-Universal-Model.
最长约 10秒,即可获得该文献文件

科研通智能强力驱动
Strongly Powered by AbleSci AI
科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
ZY完成签到 ,获得积分10
1秒前
1秒前
syyyy完成签到,获得积分20
2秒前
2秒前
英姑的应助被温柔的依霜采纳,获得10
2秒前
owoo完成签到,获得积分10
3秒前
hs完成签到 ,获得积分10
3秒前
小陈发布了新的文献求助10
3秒前
3秒前
TaoJ举报鹿呦呦呦呦的求助涉嫌违规
3秒前
迷乱完成签到 ,获得积分10
4秒前
bkagyin的应助被叫你吃布丁采纳,获得10
5秒前
6秒前
xiaolizi发布了新的文献求助30
6秒前
小巧水绿发布了新的文献求助10
6秒前
聪慧雪糕发布了新的文献求助30
7秒前
壮观雁易发布了新的文献求助10
8秒前
所所的应助被早安采纳,获得10
9秒前
天天快乐的应助被lzh采纳,获得10
9秒前
Wizard完成签到 ,获得积分10
10秒前
豆芽菜发布了新的文献求助10
10秒前
10秒前
完美世界的应助被yang采纳,获得10
10秒前
LQKK完成签到,获得积分10
11秒前
13秒前
koyo发布了新的文献求助10
13秒前
1192237414发布了新的文献求助10
13秒前
隐形曼青的应助被xhy采纳,获得10
13秒前
七树鱼完成签到,获得积分10
14秒前
14秒前
郑郑宝贝完成签到,获得积分10
14秒前
14秒前
15秒前
15秒前
15秒前
16秒前
Ascent完成签到,获得积分10
17秒前
TaoJ的应助被叫你吃布丁采纳,获得10
18秒前
18秒前
可乐发布了新的文献求助10
19秒前
高分求助中
(应助此贴封号)【重要!!请各用户(尤其是新用户)详细阅读】【科研通的精品贴汇总】 10000
拉伸高分子熔体中缠结行为的模拟和小角中子散射研究 1700
The World Atlas of Wine 9th edition 1000
奈特基础组织学:配套组织病理学(第4版)(Netter's Essential Histology With Correlated Histopathology) 800
八旗制度の研究 600
Improving the wear resistance of vehicle gear oil by oil soluble LaF3 nanoparticles and typical additives of gear oil 500
Boundaries and Bodhisattvas 500
热门求助领域 (近24小时)
化学 材料科学 医学 生物 纳米技术 计算机科学 工程类 有机化学 化学工程 生物化学 物理 复合材料 内科学 催化作用 光电子学 人工智能 细胞生物学 无机化学 物理化学 数学
热门帖子
关注 科研通微信公众号,转发送积分 8183730
求助须知:如何正确求助?哪些是违规求助? 9572975
关于积分的说明 21828453
捐赠科研通 7780887
什么是DOI,文献DOI怎么找? 3434086
关于科研通互助平台的介绍 2558276
邀请新用户注册赠送积分活动 2480557