FLFuse-Net: A fast and lightweight infrared and visible image fusion network via feature flow and edge compensation for salient information

In this paper, a fast, lightweight image fusion network, FLFuse-Net, is proposed to generate a new perspective image with identical and discriminative features from both infrared and visible images. In this network, deep convolutional features are extracted and fused synchronously through feature flow, while the edge features of the salient targets from the infrared image are compensated asynchronously. First, we design an autoencoder network structure with cross-connections for simultaneous feature extraction and fusion. In this structure, the fusion strategy is carried out through feature flow rather than by using a fixed fusion strategy, as in previous works. Second, we propose an edge compensation branch for salient information with the corresponding edge loss function to obtain the edge features of salient information from infrared images. Third, our network is designed as a lightweight network with a small number of parameters and low computational complexity, resulting in lower hardware requirements and a faster calculation speed. The experimental results confirm that the proposed FLFuse-Net outperforms the state-of-the-art fusion methods in objective and subjective assessments with very few parameters, especially on the TNO Image Fusion and NIR Scenes datasets. • In our proposed network, the information extraction and fusion are implemented simultaneously. On the one hand, it is no longer to design a fixed fusion strategy, and on the other hand, it reduces the additional computational effort caused by additional fusion strategies. • Our proposed network has been designed to be fast and lightweight. Especially, the weight sharing and information exchanging between two branches makes the network lightweight and fast. Inference speed of 512 × 512 images on RTX2080Ti takes less than 1ms. • An edge compensation branch and the corresponding edge loss function are proposed to provide more edge information for salient thermal targets from the infrared images to fused images.