An efficient frequency domain fusion network of infrared and visible images

Image fusion plays a crucial role in enhancing the quality and accuracy of semantic segmentation, which is essential for autonomous driving systems. By merging information from multiple imaging sensors or modalities, such as infrared and visible images, image fusion enriches the data and improves the perception capabilities of autonomous vehicles. However, current fusion methodologies often cannot balance model complexity, inference efficiency, and fusion accuracy simultaneously, making them difficult to implement in resource-constrained environments. In response to this, this paper presents a lightweight fusion network based on frequency transformation and deep learning techniques, leveraging wavelet transformation to fuse infrared and visible images. Concisely, the fusion model decomposes input images into different frequency sub-bands using wavelet transforms. It then efficiently fuses the multi-scale feature representations in the frequency domains with a specially designed fusion loss. Compared to traditional fusion approaches, our method not only achieves a better balance between subjective fusion quality and downstream vision tasks but also significantly improves model inference efficiency, paving the way for real-time autonomous driving systems. Extensive experiments on public datasets show that our method can achieve state-of-the-art performance while satisfying parameter efficiency in the context of image fusion and semantic segmentation tasks. Concisely, our approach is nearly 100× faster while using a model 6000× smaller in size compared to SegMIF.