An overview on spectral and spatial information fusion for hyperspectral image classification: Current trends and challenges

• A review of spectral-spatial fusion methods for hyperspectral images is presented. • Fusion methods are divided into segmentation based, feature fusion, decision fusion. • Object based methods and pixel wise ones are discussed in segmentation based fusion. • 3D feature extraction and deep learning are discussed in feature fusion. • Various complement classification methods are discussed in decision fusion. Hyperspectral images (HSIs) have a cube form containing spatial information in two dimensions and rich spectral information in the third one. The high volume of spectral bands allows discrimination between various materials with high details. Moreover, by utilizing the spatial features of image such as shape, texture and geometrical structures, the land cover discrimination will be improved. So, fusion of spectral and spatial information can significantly improve the HSI classification. In this work, the spectral-spatial information fusion methods are categorized into three main groups. The first group contains segmentation based methods where objects or super-pixels are used instead of pixels for classification or the obtained segmentation map is used for relaxation of the pixel-wise classification map. The second group consists of feature fusion methods which are divided into six sub-groups: features stacking, joint spectral-spatial feature extraction, kernel based classifiers, representation based classifiers, 3D spectral-spatial feature extraction and deep learning based classifiers. The third fusion methods are decision fusion based approaches where complementary information of several classifiers are contributed for achieving the final classification map. A review of different methods in each category, is presented. Moreover, the advantages and difficulties/disadvantages of each group are discussed. The performance of various fusion methods are assessed in terms of classification accuracy and running time using experiments on three popular hyperspectral images. The results show that the feature fusion methods although are time consuming but can provide superior classification accuracy compared to other methods. Study of this work can be very useful for all researchers interested in HSI feature extraction, fusion and classification.