Cross-Modal Contrastive Learning for Remote Sensing Image Classification

Recently, multi-modal remote sensing image (MRSI) classification has attracted increasing attention of researchers. However, classification of MRSI with limited labeled instances is still a challenging task. In this paper, a novel self-supervised cross-modal contrastive learning method is proposed for MRSI classification. Joint intra- and cross-modal contrastive learning are used to better mine multi-modal feature representations during pre-training, and the intra- and cross-modal contrastive learning objectives are jointly optimized, whereby it encourages the learned representation to be semantically consistent within and between modalities simultaneously. Moreover, a simple but effective hybrid cross-modal fusion module (HCFM) is designed in the fine-tuning stage, which could better compactly integrate complementary information across these modalities for more accurate classification. Extensive experiments are taken on four benchmark datasets (i.e., Houston 2013, Augsburg, Trento, and Berlin), and the results show that the proposed method outperforms state-of-the-art methods.