R-YOLO: A Robust Object Detector in Adverse Weather

Learning a robust object detector in adverse weather with real-time efficiency is of great importance for the visual perception task for autonomous driving systems. In this article, we propose a framework to improve the YOLO to a robust detector, denoted as R(obust)-YOLO, without the need for annotations in adverse weather. Considering the distribution gap between the normal weather images and the adverse weather images, our framework consists of an image quasi-translation network (QTNet) and a feature calibration network (FCNet) for adapting the normal weather domain to the adverse weather domain gradually. Specifically, we use the simple yet effective QTNet for generating images that inherit the annotations in the normal weather domain and interpolate the gap between the two domains. Then, in FCNet, we propose two kinds of adversarial-learning-based feature calibration modules to effectively align the feature representations in two domains in a local-to-global manner. With such a learning framework, our R-YOLO does not change the original YOLO structure, and thus it is applicable to all the YOLO-series detectors. Extensive experimental results of our R-YOLOv3, R-YOLOv5, and R-YOLOX on both the hazy and rainy datasets show that our method outperforms other detectors with dehaze/derain as the preprocessing step and other unsupervised domain adaptation (UDA)-based detectors, which confirms the effectiveness of our method on improving the robustness by only leveraging the unlabeled adverse weather images. Our code and pretrained models are available at: https://github.com/qinhongda8/R-YOLO .