Unsupervised Lesion-Aware Transfer Learning for Diabetic Retinopathy Grading in Ultra-Wide-Field Fundus Photography

Ultra-wide-field (UWF) fundus photography is a new imaging technique with providing a broader field of view images, and it has become a popular and effective tool for the screening and diagnosis for many eye diseases, such as diabetic retinopathy (DR). However, it is practically challenging to train a robust deep learning model for DR grading in UWF images, due to the limited scale of data and manual annotations. By contrast, we may find large-scale high-quality regular color fundus photography datasets in the research community, with either image-level or pixel-level annotation. In consequence, we propose an Unsupervised Lesion-aware TRAnsfer learning framework (ULTRA) for DR grading in UWF images, by leveraging a large amount of publicly well-annotated regular color fundus images. Inspired by the clinical identification of DR severity, i.e., the decision making process of ophthalmologists based on the type and number of associated lesions, we design an adversarial lesion map generator to provide the auxiliary lesion information for DR grading. A Lesion External Attention Module (LEAM) is introduced to integrate the lesion feature into the model, allowing a relative explainable DR grading. Extensive experimental results show the proposed method is superior to the state-of-the-art methods.