CSTGAN: Cycle Swin Transformer GAN for Unpaired Infrared Image Colorization

Infrared images can be captured in harsh conditions such as low light and foggy weather, which provides an effective solution for image capture throughout the day. However, the low contrast and blurred object boundaries of infrared images hinder human interpretation and the application of computer vision algorithms. Colorizing infrared images is a significant and effective method to promote infrared image understanding. Image-to-image translation methods based on generative adversarial networks are currently the main methods for colorizing infrared images. The generative adversarial network designed by Transformer overcomes the disadvantage of weak global information capture ability caused by the convolutional generative adversarial network product focusing on local features. This paper proposed a new method called Cycle Swin Transformer Generative Adversarial Networks (CSTGAN) based on Cycle-Consistent Generative Adversarial Networks. The proposed method redesigns the generator with Swin Transformer and convolution layers, and modified the discriminator and loss function. The proposed method combines the advantages of convolution and Transformer to obtain accurate mapping between infrared image domain and visible light image domain, which reduces the artifacts and distortions caused by the existing infrared image colorization methods. In addition, we collected and produced a near-infrared image colorization dataset named NIR2RGB. Extensive experimental results show that the proposed method outperforms the previous methods on the FID and KID metrics on the public datasets RGB-NIR Scene and MFNet as well as produced NIR2RGB.