Transformer-Based Masked Autoencoder With Contrastive Loss for Hyperspectral Image Classification

Recent years, in order to solve the problem of lacking accurately labeled hyperspectral image data, self-supervised learning has become an effective method for hyperspectral image classification. The core idea of self-supervised learning is to define a pretext task which helps to train the model without the labels. By exploiting both the information of the labeled and unlabeled samples, self-supervised learning shows enormous potential to handle many different tasks in the field of hyperspectral image processing. Among the vast amount of self-supervised methods, contrastive learning and masked autoencoder are well known because of their impressive performance. This article proposes a Transformer based masked autoencoder using contrastive learning (TMAC), which tries to combine these two methods and improve the performance further. TMAC has two branches, the first branch has an encoder-decoders structure, it has an encoder to capture the latent image representation of the masked hyperspectral image and two decoders where the pixel decoder aims to reconstruct the hyperspectral image at pixel-level and the feature decoder is built to extract the high-level feature of the reconstructed image. The second branch consists of a momentum encoder and a standard projection head to embed the image into the feature space. Then, by combining the output of feature decoder and the embedding vectors via contrastive learning to enhance the model’s classification performance. According to the experiments, our model shows powerful feature extraction capability and gets outstanding results on hyperspectral image datasets.