已入深夜,您辛苦了!由于当前在线用户较少,发布求助请尽量完整的填写文献信息,科研通机器人24小时在线,伴您度过漫漫科研夜!祝你早点完成任务,早点休息,好梦!

Edge Guided GANs With Multi-Scale Contrastive Learning for Semantic Image Synthesis

计算机科学 人工智能 情报检索 自然语言处理
作者
Hao Tang,Guolei Sun,Nicu Sebe,Luc Van Gool
出处
期刊:IEEE Transactions on Pattern Analysis and Machine Intelligence [Institute of Electrical and Electronics Engineers]
卷期号:45 (12): 14435-14452 被引量:1
标识
DOI:10.1109/tpami.2023.3298721
摘要

We propose a novel e dge guided g enerative a dversarial n etwork with c ontrastive learning (ECGAN) for the challenging semantic image synthesis task. Although considerable improvements have been achieved by the community in the recent period, the quality of synthesized images is far from satisfactory due to three largely unresolved challenges. 1) The semantic labels do not provide detailed structural information, making it challenging to synthesize local details and structures; 2) The widely adopted CNN operations such as convolution, down-sampling, and normalization usually cause spatial resolution loss and thus cannot fully preserve the original semantic information, leading to semantically inconsistent results (e.g., missing small objects); 3) Existing semantic image synthesis methods focus on modeling “local” semantic information from a single input semantic layout. However, they ignore “global” semantic information of multiple input semantic layouts, i.e., semantic cross-relations between pixels across different input layouts. To tackle 1), we propose to use the edge as an intermediate representation which is further adopted to guide image generation via a proposed attention guided edge transfer module. Edge information is produced by a convolutional generator and introduces detailed structure information. To tackle 2), we design an effective module to selectively highlight class-dependent feature maps according to the original semantic layout to preserve the semantic information. To tackle 3), inspired by current methods in contrastive learning, we propose a novel contrastive learning method, which aims to enforce pixel embeddings belonging to the same semantic class to generate more similar image content than those from different classes. We further propose a novel multi-scale contrastive learning method that aims to push same-class features from different scales closer together being able to capture more semantic relations by explicitly exploring the structures of labeled pixels from multiple input semantic layouts from different scales. Experiments on three challenging datasets show that our methods achieve significantly better results than state-of-the-art approaches. The source code is available at https://github.com/Ha0Tang/ECGAN .
最长约 10秒,即可获得该文献文件

科研通智能强力驱动
Strongly Powered by AbleSci AI
更新
大幅提高文件上传限制,最高150M (2024-4-1)

科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
1秒前
yk完成签到 ,获得积分10
4秒前
快乐抽屉发布了新的文献求助10
5秒前
Kashing完成签到 ,获得积分10
6秒前
7秒前
8秒前
9秒前
小马甲应助小米采纳,获得10
10秒前
hui_L发布了新的文献求助10
10秒前
bubble完成签到 ,获得积分10
10秒前
自然的依丝完成签到,获得积分20
11秒前
14秒前
albert666完成签到,获得积分10
15秒前
魏阳虹完成签到 ,获得积分10
18秒前
科yt完成签到 ,获得积分10
18秒前
敏感的钢铁侠完成签到,获得积分10
19秒前
我想开兰博完成签到 ,获得积分10
25秒前
清新的音响完成签到 ,获得积分10
26秒前
小猫爱吃鱼完成签到,获得积分20
27秒前
hg秀秀完成签到 ,获得积分10
27秒前
orixero应助liweiDr采纳,获得10
28秒前
细腻的秋天完成签到 ,获得积分10
29秒前
Singularity应助WLL采纳,获得10
29秒前
星回完成签到,获得积分10
31秒前
31秒前
32秒前
快乐抽屉完成签到,获得积分20
33秒前
小叮当完成签到,获得积分10
34秒前
35秒前
DrWen完成签到 ,获得积分20
36秒前
修稳定ah26发布了新的文献求助10
36秒前
六个核桃发布了新的文献求助10
36秒前
轻歌水越完成签到 ,获得积分10
38秒前
旺仔先生完成签到,获得积分0
38秒前
Zhang_Yakun完成签到 ,获得积分10
39秒前
Bio完成签到,获得积分10
43秒前
43秒前
JamesPei应助ANON_TOKYO采纳,获得10
46秒前
悦耳代亦完成签到 ,获得积分0
47秒前
47秒前
高分求助中
Kinetics of the Esterification Between 2-[(4-hydroxybutoxy)carbonyl] Benzoic Acid with 1,4-Butanediol: Tetrabutyl Orthotitanate as Catalyst 1000
The Young builders of New china : the visit of the delegation of the WFDY to the Chinese People's Republic 1000
Rechtsphilosophie 1000
Handbook of Qualitative Cross-Cultural Research Methods 600
Chen Hansheng: China’s Last Romantic Revolutionary 500
Mantiden: Faszinierende Lauerjäger Faszinierende Lauerjäger 500
PraxisRatgeber: Mantiden: Faszinierende Lauerjäger 500
热门求助领域 (近24小时)
化学 医学 生物 材料科学 工程类 有机化学 生物化学 物理 内科学 纳米技术 计算机科学 化学工程 复合材料 基因 遗传学 催化作用 物理化学 免疫学 量子力学 细胞生物学
热门帖子
关注 科研通微信公众号,转发送积分 3139360
求助须知:如何正确求助?哪些是违规求助? 2790295
关于积分的说明 7794749
捐赠科研通 2446704
什么是DOI,文献DOI怎么找? 1301351
科研通“疑难数据库(出版商)”最低求助积分说明 626134
版权声明 601123