Cross-Modal Adaptation for Object Detection in Thermal InfraRed Remote Sensing Imagery

Modern Thermal InfraRed (TIR) technology has been proven highly significant in Remote Sensing Imagery (RSI). Currently, multimodal RSI object detection based on RGB-TIR image pairs has attracted widespread research. However, capturing features in the TIR domain poses a challenge, as existing object detectors heavily focus on chromatic information in the RGB domain. Furthermore, the quality of RGB images can be influenced by complex environmental conditions, limiting the practicality of multimodal detection. In this paper, we introduce Cross-Modal-YOLO (CM-YOLO), a lightweight yet effective object detector specifically designed for TIR remote sensing images. CM-YOLO employs cross-modal adaptation to enhance the awareness of TIR-RGB modality translation. Specifically, we leverage a Prior Modality Translator (PMT) to learn the InfraRed-Visible (IV) features, which are incorporated into the detection backbone using our IV-Gate modules. Experimental results on the VEDAI dataset demonstrate that CM-YOLO significantly outperforms conventional methods. Moreover, CM-YOLO exhibits a strong generalization ability for TIR-based object detection in urban scenes on the FLIR dataset.