Multi-Scale Transformer Network With Edge-Aware Pre-Training for Cross-Modality MR Image Synthesis

基本事实 人工智能 计算机科学 编码器 模态(人机交互) 自编码 模式识别(心理学) GSM演进的增强数据速率 深度学习 计算机视觉 操作系统
作者
Yonghao Li,Tao Zhou,Kelei He,Yi Zhou,Dinggang Shen
出处
期刊:IEEE Transactions on Medical Imaging [Institute of Electrical and Electronics Engineers]
卷期号:42 (11): 3395-3407 被引量:21
标识
DOI:10.1109/tmi.2023.3288001
摘要

Cross-modality magnetic resonance (MR) image synthesis can be used to generate missing modalities from given ones. Existing (supervised learning) methods often require a large number of paired multi-modal data to train an effective synthesis model. However, it is often challenging to obtain sufficient paired data for supervised training. In reality, we often have a small number of paired data while a large number of unpaired data. To take advantage of both paired and unpaired data, in this paper, we propose a Multi-scale Transformer Network (MT-Net) with edge-aware pre-training for cross-modality MR image synthesis. Specifically, an Edge-preserving Masked AutoEncoder (Edge-MAE) is first pre-trained in a self-supervised manner to simultaneously perform 1) image imputation for randomly masked patches in each image and 2) whole edge map estimation, which effectively learns both contextual and structural information. Besides, a novel patch-wise loss is proposed to enhance the performance of Edge-MAE by treating different masked patches differently according to the difficulties of their respective imputations. Based on this proposed pre-training, in the subsequent fine-tuning stage, a Dual-scale Selective Fusion (DSF) module is designed (in our MT-Net) to synthesize missing-modality images by integrating multi-scale features extracted from the encoder of the pre-trained Edge-MAE. Furthermore, this pre-trained encoder is also employed to extract high-level features from the synthesized image and corresponding ground-truth image, which are required to be similar (consistent) in the training. Experimental results show that our MT-Net achieves comparable performance to the competing methods even using 70% of all available paired data. Our code will be released at https://github.com/lyhkevin/MT-Net .
最长约 10秒,即可获得该文献文件

科研通智能强力驱动
Strongly Powered by AbleSci AI
更新
PDF的下载单位、IP信息已删除 (2025-6-4)

科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
厘米完成签到 ,获得积分10
1秒前
微笑的小霸王完成签到,获得积分10
2秒前
HK完成签到 ,获得积分10
4秒前
超级的千青完成签到 ,获得积分10
5秒前
量子星尘发布了新的文献求助150
5秒前
取法乎上完成签到 ,获得积分10
7秒前
楚楚完成签到 ,获得积分10
9秒前
月上柳梢头A1完成签到,获得积分10
9秒前
萧然完成签到,获得积分10
13秒前
自信的访云完成签到,获得积分10
14秒前
nusiew完成签到,获得积分10
15秒前
li完成签到 ,获得积分10
16秒前
clm完成签到 ,获得积分10
17秒前
tangzanwayne完成签到,获得积分10
18秒前
哈哈完成签到 ,获得积分10
21秒前
一亩蔬菜完成签到,获得积分10
25秒前
advance完成签到,获得积分10
31秒前
yier完成签到,获得积分10
33秒前
芒果布丁完成签到 ,获得积分10
33秒前
金枪鱼子发布了新的文献求助10
34秒前
高高的天亦完成签到 ,获得积分10
37秒前
小杨完成签到 ,获得积分10
38秒前
40秒前
量子星尘发布了新的文献求助10
40秒前
甜甜的满天完成签到,获得积分10
44秒前
哈哈哈发布了新的文献求助10
45秒前
老八的嘴完成签到,获得积分10
45秒前
发发完成签到,获得积分10
46秒前
她的城完成签到,获得积分0
47秒前
gao完成签到 ,获得积分10
49秒前
一行白鹭上青天完成签到 ,获得积分10
50秒前
51秒前
tzjz_zrz完成签到,获得积分10
53秒前
欢呼妙菱完成签到,获得积分10
55秒前
进击的巨人完成签到 ,获得积分10
56秒前
我就想看看文献完成签到 ,获得积分10
58秒前
逍遥呱呱完成签到 ,获得积分10
1分钟前
体贴的叛逆者完成签到,获得积分10
1分钟前
lindahappy关注了科研通微信公众号
1分钟前
某只橘猫君完成签到,获得积分10
1分钟前
高分求助中
【提示信息,请勿应助】关于scihub 10000
A new approach to the extrapolation of accelerated life test data 1000
Coking simulation aids on-stream time 450
北师大毕业论文 基于可调谐半导体激光吸收光谱技术泄漏气体检测系统的研究 390
Phylogenetic study of the order Polydesmida (Myriapoda: Diplopoda) 370
Robot-supported joining of reinforcement textiles with one-sided sewing heads 360
Novel Preparation of Chitin Nanocrystals by H2SO4 and H3PO4 Hydrolysis Followed by High-Pressure Water Jet Treatments 300
热门求助领域 (近24小时)
化学 材料科学 医学 生物 工程类 有机化学 生物化学 物理 内科学 纳米技术 计算机科学 化学工程 复合材料 遗传学 基因 物理化学 催化作用 冶金 细胞生物学 免疫学
热门帖子
关注 科研通微信公众号,转发送积分 4015640
求助须知:如何正确求助?哪些是违规求助? 3555625
关于积分的说明 11318138
捐赠科研通 3288796
什么是DOI,文献DOI怎么找? 1812284
邀请新用户注册赠送积分活动 887882
科研通“疑难数据库(出版商)”最低求助积分说明 812015