Multimodal super-resolution reconstruction of infrared and visible images via deep learning

In this paper, we propose a deep-learning-based infrared-visible images fusion method based on encoder-decoder architecture. The image fusion task is reformulated as a problem of maintaining the structure and intensity ratio of the infrared-visible image. The corresponding loss function is designed to expand the weight difference between the thermal target and the background. In addition, a single image super-resolution reconstruction based on a regression network is introduced to address the issue that traditional network mapping functions are not suitable for natural scenes. The forward generation and reverse regression models are considered to reduce the irrelevant function mapping space and approach the ideal scene data through double mapping constraints. Compared with other state-of-the-art approaches, our experimental results achieve superior performance in terms of both visual effects and objective assessments. In addition, it can stably provide high-resolution reconstruction results consistent with human visual observation while bridging the resolution gap between the infrared-visible images.