MGIML: Cancer Grading With Incomplete Radiology-Pathology Data via Memory Learning and Gradient Homogenization

分级(工程) 均质化(气候) 放射科 计算机科学 病理 人工智能 医学 医学物理学 生物 生物多样性 生态学
作者
Pengyu Wang,Huaqi Zhang,Meilu Zhu,Xi Jiang,Jing Qin,Yixuan Yuan
出处
期刊:IEEE Transactions on Medical Imaging [Institute of Electrical and Electronics Engineers]
卷期号:43 (6): 2113-2124
标识
DOI:10.1109/tmi.2024.3355142
摘要

Taking advantage of multi-modal radiology-pathology data with complementary clinical information for cancer grading is helpful for doctors to improve diagnosis efficiency and accuracy. However, radiology and pathology data have distinct acquisition difficulties and costs, which leads to incomplete-modality data being common in applications. In this work, we propose a Memory-and Gradient-guided Incomplete Modal-modal Learning (MGIML) framework for cancer grading with incomplete radiology-pathology data. Firstly, to remedy missing-modality information, we propose a Memory-driven Hetero-modality Complement (MH-Complete) scheme, which constructs modal-specific memory banks constrained by a coarse-grained memory boosting (CMB) loss to record generic radiology and pathology feature patterns, and develops a cross-modal memory reading strategy enhanced by a fine-grained memory consistency (FMC) loss to take missing-modality information from well-stored memories. Secondly, as gradient conflicts exist between missing-modality situations, we propose a Rotation-driven Gradient Homogenization (RG-Homogenize) scheme, which estimates instance-specific rotation matrices to smoothly change the feature-level gradient directions, and computes confidence-guided homogenization weights to dynamically balance gradient magnitudes. By simultaneously mitigating gradient direction and magnitude conflicts, this scheme well avoids the negative transfer and optimization imbalance problems. Extensive experiments on CPTAC-UCEC and CPTAC-PDA datasets show that the proposed MGIML framework performs favorably against state-of-the-art multi-modal methods on missing-modality situations.
最长约 10秒,即可获得该文献文件

科研通智能强力驱动
Strongly Powered by AbleSci AI
更新
大幅提高文件上传限制,最高150M (2024-4-1)

科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
SciGPT应助Tay采纳,获得10
刚刚
肉肉完成签到,获得积分10
1秒前
3秒前
向日葵发布了新的文献求助10
4秒前
4秒前
4秒前
云喆瑜瑾完成签到,获得积分10
5秒前
rz3790完成签到,获得积分20
6秒前
6秒前
fairy完成签到,获得积分20
7秒前
7秒前
充电宝应助星星采纳,获得10
7秒前
7秒前
7秒前
FashionBoy应助不吃折耳根采纳,获得10
7秒前
眰恦发布了新的文献求助10
8秒前
爆米花应助123采纳,获得10
9秒前
冰冰发布了新的文献求助10
10秒前
10秒前
10秒前
完美世界应助LSS采纳,获得10
10秒前
10秒前
笑笑发布了新的文献求助10
10秒前
10秒前
hhh发布了新的文献求助10
11秒前
nuli完成签到 ,获得积分10
11秒前
机智的曼易完成签到 ,获得积分10
11秒前
马如辰完成签到,获得积分20
12秒前
12秒前
KKK完成签到,获得积分10
12秒前
Eve发布了新的文献求助10
13秒前
薰硝壤应助朴素雁凡采纳,获得10
13秒前
14秒前
可爱的函函应助勤劳黄豆采纳,获得10
14秒前
Ray发布了新的文献求助10
14秒前
叽哩咕噜发布了新的文献求助30
14秒前
彭于彦祖应助shae_2022采纳,获得100
14秒前
15秒前
Young完成签到,获得积分10
15秒前
LORI发布了新的文献求助10
15秒前
高分求助中
Sustainability in Tides Chemistry 2000
Microlepidoptera Palaearctica, Volumes 1 and 3 - 13 (12-Volume Set) [German] 1122
Дружба 友好报 (1957-1958) 1000
The Data Economy: Tools and Applications 1000
A Dissection Guide & Atlas to the Rabbit 600
中国心血管健康与疾病报告2023(要完整的报告) 500
Ожившие листья и блуждающие цветы. Практическое руководство по содержанию богомолов [Alive leaves and wandering flowers. A practical guide for keeping praying mantises] 500
热门求助领域 (近24小时)
化学 医学 生物 材料科学 工程类 有机化学 生物化学 物理 内科学 纳米技术 计算机科学 化学工程 复合材料 基因 遗传学 催化作用 物理化学 免疫学 量子力学 细胞生物学
热门帖子
关注 科研通微信公众号,转发送积分 3102382
求助须知:如何正确求助?哪些是违规求助? 2753656
关于积分的说明 7624478
捐赠科研通 2406188
什么是DOI,文献DOI怎么找? 1276717
科研通“疑难数据库(出版商)”最低求助积分说明 616918
版权声明 599103