A Two-Stream Stacked Autoencoder With Inter-Class Separability for Bilinear Hyperspectral Unmixing

Deep learning-based hyperspectral unmixing (HU) is getting increasing attention in the field of remote sensing, aiming at endmember extraction and abundance estimation at pixel scale. However, many existing deep learning-based unmixing methods base on linear mixing models, neglecting complex nonlinear light scattering interactions. Furthermore, these methods often treat all spectral bands indiscriminately, ignoring characteristic differences between endmembers, hampering endmember separation. To address these issues, we present BU-Net, a novel approach for HU based on the generalized bilinear mixing model (GBM), which is a two-stream stacked autoencoder architecture designed to enhance inter-class separability. In the encoder, we employ 3D convolutions with multiple receptive field to extract multiscale spatial and spectral features simultaneously. Additionally, we design a novel band selection based on inter-class separability (BSICS), which identifies bands with inter-class separability (BICS) and the obtained bands are taken as an additional stream for improving performance. In the decoder, BU-Net develops a two-stream structure encompassing linear and bilinear elements, aligning with the theoretical components and constraints of GBM. To further enhance separability between endmembers during training, we use the spectral angle distance between BICS and its reconstruction as a loss regularization term. Moreover, we utilize materials' representative pixels obtained in the process of BSICS to initialize endmembers, which offers effective guidance for modeling the spectral properties. Experimental results on synthetic and real hyperspectral datasets show that our method outperforms state-of-the-art methods. This novel approach addresses limitations of linear mixing models while leveraging deep learning to improve accuracy of HU.