Multiscale spectral-spatial feature learning for hyperspectral image classification

• This work demonstrates that it is crucial to use multiscale features and spectral-spatial information for HSI classification, as proven by the excellent performances of the proposed model on various datasets. • We created a hierarchical feature fusion model, FFM, for accurate ground object classification with HSIs. In addition, the proposed model, MulNet, is effective and easy to implement. • MulNet can delicately handle spectral heterogeneity, thus enjoying good generalizability and resulting in high classification accuracy for different HSIs. Hyperspectral image (HSI) classification is a prevalent topic in the remote sensing image processing community. Recently, deep learning has been successfully applied to this area. However, there is still room for improvement. Since HSIs provide rich spectral information while being prone to spectral heterogeneity that damages the classification accuracy, we propose a multiscale spectral-spatial feature learning network (MulNet), which aptly handles the information given by HSIs. Our model is a hybrid model combined with a 3-Dimensional Residual Network (3DResNet), a Feature Fusion Module (FFM), and a Recurrent Neural Network (RNN). 3DResNet encodes the original HSIs and learns local spectral-spatial features at multiple scales, which are upsampled by different ratios and aggregated by FFM. Afterward, the fused features are fed sequentially to the RNN, which exploits HSI’s relations and broad contexts to produce discriminative features for better classification. Experiments on five real-world datasets using random and disjointed samples demonstrate the efficacy and efficiency of the proposed networks. It outperforms several classic and newly published spectral-spatial classifiers for HSIs.