A Hybrid Spectral Attention-Enabled Multi-Scale Spatial-Spectral Learning Network for Hyperspectral Image Super Resolution

Hyperspectral image super resolution (HSI-SR) has become an essential step of data preprocessing for tasks such as classification and change detection in remote sensing. For HSI-SR tasks, the state-of-the-art methods lie in how to learn effective spatial and spectral characteristics. More deeply mining the shallow and deep spatial-spectral features is vital for the performance improvement of HSI-SR. Hereby, we provide an end-to-end multi-scale spatial-spectral network (M3SN) driven by a hybrid spectral attention mechanism for HSI-SR, aiming to fully dig shallow and deep spatial-spectral characteristics. Precisely, considering the importance of shallow spatial-spectral features at early stages, a multi-scale information block (MSIB) consisting of a 3D convolution, three parallel 2D convolutions with different scale sizes, and SE-Net is first designed to extract informative multi-scale shallow spatial-spectral features and recalibrate the channel weights of features. Then a dual-path multi-scale spatial-spectral feature block (DMFB) is set up to explore the deep spatial and spatial-spectral features. While one path using 2D convolutions extracts spatial features, the other path employing 3D-Res2Net as well as an updated hybrid spectral attention mechanism (HSAM) module mines multi-scale deep spatial-spectral features. Finally, we design a multi-scale fusion block (MSFB) based on the channel reduction-scaling operation to fuse the extracted hierarchical feature maps for the final reconstruction. It is demonstrated that M3SN outperforms existing methods by extensive experiments on four publicly available hyperspectral datasets.