计算机科学
湍流
解耦(概率)
变压器
编码(集合论)
人工智能
计算机工程
算法
工程类
物理
电气工程
集合(抽象数据类型)
控制工程
电压
热力学
程序设计语言
作者
Xingguang Zhang,Zhiyuan Mao,Nicholas Chimitt,Stanley H. Chan
出处
期刊:IEEE transactions on computational imaging
日期:2024-01-01
卷期号:10: 115-128
被引量:10
标识
DOI:10.1109/tci.2024.3354421
摘要
Restoring images distorted by atmospheric turbulence is a ubiquitous problem in long-range imaging applications. While existing deep-learning-based methods have demonstrated promising results in specific testing conditions, they suffer from three limitations: (1) lack of generalization capability from synthetic training data to real turbulence data; (2) failure to scale, hence causing memory and speed challenges when extending the idea to a large number of frames; (3) lack of a fast and accurate simulator to generate data for training neural networks. In this paper, we introduce the turbulence mitigation transformer (TMT) that explicitly addresses these issues. TMT brings three contributions: Firstly, TMT explicitly uses turbulence physics by decoupling the turbulence degradation and introducing a multi-scale loss for removing distortion, thus improving effectiveness. Secondly, TMT presents a new attention module along the temporal axis to extract extra features efficiently, thus improving memory and speed. Thirdly, TMT introduces a new simulator based on the Fourier sampler, temporal correlation, and flexible kernel size, thus improving our capability to synthesize better training data. TMT outperforms state-of-the-art video restoration models, especially in generalizing from synthetic to real turbulence data. Code, videos, and datasets are available at https://xg416.github.io/TMT .
科研通智能强力驱动
Strongly Powered by AbleSci AI