RGB颜色模型
人工智能
计算机科学
依赖关系(UML)
计算机视觉
降噪
噪音(视频)
模式识别(心理学)
图像(数学)
作者
Jun Wang,Peilin Liu,Fei Wen
出处
期刊:IEEE transactions on image processing
[Institute of Electrical and Electronics Engineers]
日期:2022-12-07
卷期号:32: 159-174
被引量:5
标识
DOI:10.1109/tip.2022.3226419
摘要
Due to the imaging mechanism of time-of-flight (ToF) sensors, the captured depth images usually suffer from severe noise and degradation. Though many RGB-guided methods have been proposed for depth image enhancement in the past few years, yet the enhancement performance on real-world depth images is still largely unsatisfactory. Two main reasons are the complexity of realistic noise and degradation in depth images, and the difficulty in collecting noise-clean pairs for supervised enhancement learning. This work aims to develop a self-supervised learning method for RGB-guided depth image enhancement, which does not require any noisy-clean pairs but can significantly boost the enhancement performance on real-world noisy depth images. To this end, we exploit the dependency between RGB and depth images to self-supervise the learning of the enhancement model. It is achieved by maximizing the cross-modal dependency between RGB and depth to promote the enhanced depth having dependency with the RGB of the same scene as much as possible. Furthermore, we augment the cross-modal dependency maximization formulation based on the optimal transport theory to achieve further performance improvement. Experimental results on both synthetic and real-world data demonstrate that our method can significantly outperform existing state-of-the-art methods on depth denoising, multi-path interference suppression, and hole filling. Particularly, our method shows remarkable superiority over existing ones on real-world data in handling various realistic complex degradation. Code is available at https://github.com/wjcyt/SRDE.
科研通智能强力驱动
Strongly Powered by AbleSci AI