Hyperspectral image classification using spectral-spatial LSTMs

Abstract In this paper, we propose a hyperspectral image (HSI) classification method using spectral-spatial long short term memory (LSTM) networks. Specifically, for each pixel, we feed its spectral values in different channels into Spectral LSTM one by one to learn the spectral feature. Meanwhile, we firstly use principle component analysis (PCA) to extract the first principle component from a HSI, and then select local image patches centered at each pixel from it. After that, we feed the row vectors of each image patch into Spatial LSTM one by one to learn the spatial feature for the center pixel. In the classification stage, the spectral and spatial features of each pixel are fed into softmax classifiers respectively to derive two different results, and a decision fusion strategy is further used to obtain a joint spectral-spatial results. Experimental results on three widely used HSIs (i.e., Indian Pines, Pavia University, and Kennedy Space Center) show that our method can improve the classification accuracy by at least 2.69%, 1.53% and 1.08% compared to other state-of-the-art methods.