Unsupervised Encoder–Decoder Network Under Spatial and Spectral Guidance for Hyperspectral and Multispectral Image Fusion

Due to the limitations of hyperspectral optical imaging, hyperspectral images have a dilemma between spectral and spatial resolutions. Hyperspectral and multispectral image (HSI-MSI) fusion, which combines a low-spatial-resolution hyperspectral image (LR-HSI) and a high-spatial-resolution multispectral image (HR-MSI), can generate a high-spatial-resolution hyperspectral image (HR-HSI). In existing methods for hyperspectral and multispectral fusion, correlation between spectral and spatial domains in HSIs is mostly neglected. To address this issue, an unsupervised encoder-decoder network under spatial and spectral guidance for hyperspectral and multispectral image fusion (uEDSSG) was proposed in this study. To learn more accurate abundances of a LR-HSI and a HR-MSI, multi-hierarchical encoders under spatial and spectral guidance were designed to extract multi-hierarchical fused features from the LR-HSI and HR-MSI with the guidance of the HR-MSI and LR-HSI, respectively. In the new method, deep coupling of the point spread function (PSF) or spectral response function (SRF) and edge of the HSIs was designed to maintain the spatial and spectral details of the HR-HSI; a spatial-spectral constraint was constructed to establish the relationship of the HSIs. Both visual and quantitative evaluation results of experiments based on both synthetic and real datasets showed that the proposed method outperformed seven common methods. The results suggest that the new method by maintaining the correlation between spectral and spatial domains can improve the result of HSI-MSI fusion.