Attention in Attention for Hyperspectral With High Spatial Resolution (H) Image Classification

An inevitable trend of hyperspectral remote sensing has been toward hyperspectral with high spatial resolution (H2) images. However, the higher resolution also brings higher spatial/spectral heterogeneity of surface features, which increases the difficulty of fine classification. Fully using global spatial–spectral features and contextual information is an effective method to alleviate spatial/spectral heterogeneity. Recently, to extract global spatial–spectral features with long-range dependencies, the self-attention mechanism has been widely used in H2 image classification and has achieved excellent results. As is well known, the simultaneous use of spatial and spectral information has always been a key aspect of hyperspectral image (HSI) processing; however, the current spatial and spectral attention modules only focus on the spatial and spectral features separately. This prevents further improvement in network performance, especially when the sample size is small. Therefore, a spatial–spectral attention-in-attention network (S2AiANet) is proposed, which solves the problem of the current spatial–spectral attention maps only focusing on single features through the spatial–spectral attention-in-attention (S2AiA) module. In addition, a multiscale attention (MSA) module is proposed to enhance the network's adaptability to various complex scenarios. The experiments on two H2 datasets and one classic HSI dataset demonstrate that S2AiANet can achieve a significant performance improvement compared with the state-of-the-art hyperspectral classifiers.