Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction

Hyperspectral image (HSI) reconstruction aims to recover the 3D spatial-spectral signal from a 2D measurement in the coded aperture snapshot spectral imaging (CASSI) system.The HSI representations are highly similar and correlated across the spectral dimension.Modeling the inter-spectra interactions is beneficial for HSI reconstruction.However, existing CNN-based methods show limitations in capturing spectral-wise similarity and long-range dependencies.Besides, the HSI information is modulated by a coded aperture (physical mask) in CASSI.Nonetheless, current algorithms have not fully explored the guidance effect of the mask for HSI restoration.In this paper, we propose a novel framework, Mask-guided Spectral-wise Transformer (MST), for HSI reconstruction.Specifically, we present a Spectral-wise Multi-head Self-Attention (S-MSA) that treats each spectral feature as a token and calculates self-attention along the spectral dimension.In addition, we customize a Mask-guided Mechanism (MM) that directs S-MSA to pay attention to spatial regions with high-fidelity spectral representations.Extensive experiments show that our MST significantly outperforms state-of-the-art (SOTA) methods on simulation and real HSI datasets while requiring dramatically cheaper computational and memory costs.