Self-Supervised Learning for Real-World Super-Resolution from Dual and Multiple Zoomed Observations

人工智能 计算机科学 计算机视觉 基本事实 图像(数学) 特征(语言学) 图像分辨率 超分辨率 对偶(语法数字) 模式识别(心理学) 艺术 文学类 哲学 语言学
作者
Zhilu Zhang,Ruohao Wang,Hongzhi Zhang,Wangmeng Zuo
出处
期刊:IEEE Transactions on Pattern Analysis and Machine Intelligence [IEEE Computer Society]
卷期号:: 1-13 被引量:1
标识
DOI:10.1109/tpami.2024.3379736
摘要

In this paper, we consider two challenging issues in reference-based super-resolution (RefSR) for smartphone, (i) how to choose a proper reference image, and (ii) how to learn RefSR in a self-supervised manner. Particularly, we propose a novel self-supervised learning approach for real-world RefSR from observations at dual and multiple camera zooms. Firstly, considering the popularity of multiple cameras in modern smartphones, the more zoomed (telephoto) image can be naturally leveraged as the reference to guide the super-resolution (SR) of the lesser zoomed (ultra-wide) image, which gives us a chance to learn a deep network that performs SR from the dual zoomed observations (DZSR). Secondly, for self-supervised learning of DZSR, we take the telephoto image instead of an additional high-resolution image as the supervision information, and select a center patch from it as the reference to super-resolve the corresponding ultra-wide image patch. To mitigate the effect of the misalignment between ultra-wide low-resolution (LR) patch and telephoto ground-truth (GT) image during training, we first adopt patch-based optical flow alignment to obtain the warped LR, then further design an auxiliary-LR to guide the deforming of the warped LR features. To generate visually pleasing results, we present local overlapped sliced Wasserstein loss to better represent the perceptual difference between GT and output in the feature space. During testing, DZSR can be directly deployed to super-solve the whole ultra-wide image with the reference of the telephoto image. In addition, we further take multiple zoomed observations to explore self-supervised RefSR, and present a progressive fusion scheme for the effective utilization of reference images. Experiments show that our methods achieve better quantitative and qualitative performance against state-of-the-arts. The code and pre-trained models will be publicly available.
最长约 10秒,即可获得该文献文件

科研通智能强力驱动
Strongly Powered by AbleSci AI
科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
1秒前
miemie完成签到,获得积分10
1秒前
Eidden_Zhang发布了新的文献求助10
2秒前
gongjie完成签到 ,获得积分10
2秒前
2秒前
K红豆完成签到,获得积分10
2秒前
3秒前
vary完成签到,获得积分20
3秒前
诗蕊发布了新的文献求助10
5秒前
大个的应助被Gxr采纳,获得10
5秒前
5秒前
che66完成签到,获得积分20
5秒前
wanci的应助被单杀是采纳,获得10
6秒前
6秒前
ansi完成签到,获得积分10
6秒前
赘婿的应助被老实的盼芙采纳,获得10
7秒前
xinyuanjia的应助被akakns采纳,获得10
8秒前
Chow完成签到,获得积分10
8秒前
小马甲的应助被先玉梅采纳,获得10
8秒前
8秒前
8秒前
渡人舟的应助被踏实嫣然采纳,获得150
9秒前
DLT完成签到 ,获得积分10
9秒前
9秒前
Tao完成签到,获得积分10
10秒前
10秒前
小巧紫蓝发布了新的文献求助10
10秒前
10秒前
上官若男的应助被guaguawa采纳,获得10
11秒前
田様的应助被月明星稀采纳,获得20
12秒前
12秒前
12秒前
13秒前
阔达不凡完成签到,获得积分10
13秒前
14秒前
14秒前
斯文败类的应助被WR采纳,获得10
14秒前
星辰大海的应助被kln0403采纳,获得10
15秒前
Gxr发布了新的文献求助10
15秒前
Again发布了新的文献求助20
16秒前
高分求助中
(应助此贴封号)【重要!!请各用户(尤其是新用户)详细阅读】【科研通的精品贴汇总】 10000
拉伸高分子熔体中缠结行为的模拟和小角中子散射研究 1500
The World Atlas of Wine 9th edition 1000
奈特基础组织学:配套组织病理学(第4版)(Netter's Essential Histology With Correlated Histopathology) 800
John McDowells Bildungstheorie 800
八旗制度の研究 600
Analyses de reconnaissances de dettes passées devant les échevins d'Ypres (1249-1291) 600
热门求助领域 (近24小时)
化学 材料科学 医学 生物 纳米技术 计算机科学 工程类 有机化学 化学工程 生物化学 物理 复合材料 内科学 催化作用 光电子学 人工智能 细胞生物学 无机化学 物理化学 数学
热门帖子
关注 科研通微信公众号,转发送积分 8163407
求助须知:如何正确求助?哪些是违规求助? 9557533
关于积分的说明 21740746
捐赠科研通 7754041
什么是DOI,文献DOI怎么找? 3428210
关于科研通互助平台的介绍 2552932
邀请新用户注册赠送积分活动 2472896