An Asymptotic Multiscale Symmetric Fusion Network for Hyperspectral and Multispectral Image Fusion

Despite the high spectral resolution and abundant information of hyperspectral images (HSI), their spatial resolution is relatively low due to limitations in sensor technology. Sensors often need to sacrifice some spatial resolution to ensure accurate light energy measurement when pursuing high spectral resolution. This trade-off results in HSI’s inability to capture fine spatial details, thereby limiting its application in scenarios requiring high-precision spatial information. HSI and multispectral images (MSI) fusion is a commonly used technique for generating high-resolution HSI (HR-HSI). However, many deep learning-based HSI-MSI fusion algorithms ignore correlation and multi-scale information between input images. To address this issue, we propose an asymptotic multi-scale symmetric fusion network (AMSF-Net) for hyperspectral and multispectral image fusion. AMSF-Net consists of two parts: the multi-level feature fusion (MFF) module and the progressive cross-scale spatial perception (PCP) module. The MFF module uses multi-stream feature extraction branches to perform information interaction between HSI and MSI at the same scale layer by layer, compensating for the spatial details lacking in HSI and the spectral details absent in MSI. The PCP module combines the input and output features of MFF, utilizes multi-scale bidirectional strip convolution and deep convolution to further refine edge features, and reconstructs HR-HSI by learning the features of different expansion roll branches by connecting across scales. Comparative experiments with several state-of-the-art HSI-MSI fusion algorithms on four publicly available datasets, CAVE, Chikusei, Houston and WorldView-3 are conducted to validate the effectiveness and superiority of AMSF-Net. On the Chikusei dataset, improvements were 9.1%, 12.5%, and 5.1%, respectively, on the indicators RMSE, ERGAS, and SAM, compared to the suboptimal method.