Ultra-Lightweight Feature-Compressed Multi-Head Self-Attention Learning Networks for Hyperspectral Image Classification

Vision transformers are widely used in hyperspectral image classification, with their core feature extractor being self-attention. Self-attention has a wider receptive field than convolution. However, existing vision transformers for the classification of hyperspectral images (HSIs) with a large number of bands generally suffer from high computational complexity and a large number of parameter requirements. In this paper, we propose an Ultra-lightweight Feature-compressed Multi-head Self-attention Learning Network (UFMS-LN), which mainly consists of a novel Compressed Feature Multi-Head Self-Attention (CF-MHSA), a Spatial Feature Enhancement- Enhancing Transformation Reduction (SFE-ETR) and a Spatial-spectral Hybridization-Receptive Field Attention Convolutional operation (SH-RFAConv). By effectively compressing feature maps in spatial-spectral dimensions, CF-MHSA achieves the same feature extraction capabilities as state-of-the-art self-attention mechanisms, and its floating-point operations (FLOPs) and parameters are two orders of magnitude lower than state-of-the-art self-attention mechanisms. SH-RFAConv is designed to emphasize local features, which have the ability to extract both spatial-spectral features simultaneously and have a wider receptive field than traditional convolutional operations. Furthermore, SFE-ETR is a preprocessing module for UFMS-LN that combines global spatial feature enhancement methods with Enhancing Transformation Reduction (ETR). Extensive experiments conducted on four benchmark HSI datasets have shown that this method achieves superior results compared to existing state-of-the-art HSI classification networks.