GNN-fused CapsNet with multi-head prediction for diabetic retinopathy grading

Diabetic retinopathy (DR) is a prevalent complication of diabetes, affecting a substantial number of individuals worldwide and being a leading cause of blindness. The accurate and automated detection of DR is crucial for effectively managing symptoms such as vision loss and blindness. Recently, there has been significant interest in exploring the applicability of CapsNet for DR grading regarding its success in various vision tasks. However, the performance of traditional CapsNet in DR grading is constrained by the insufficient utilization of capsule features during the training phase. To enhance its performance, this paper proposes a hybrid neural network model called graph neural network (GNN)-fused CapsNet (GF-CapsNet) for DR grading. The model combines various components including ResNet-18 for feature extraction via transfer learning, a PrimaryCaps layer for encoding capsule features, and multi-head prediction that uses GNN-based feature fusion and transformation. Experimental results obtained from two public datasets (Kaggle APTOS 2019 and IDRiD) demonstrate that GF-CapsNet outperforms traditional CapsNet and several other state-of-the-art methods in terms of capturing DR lesions and grading DR. In addition, an investigation into the internal routing process demonstrates that our method mitigates the potential misassignment problem associated with traditional CapsNet. Moreover, the use of the class activation mapping technique for feature map visualization provides an explanation of our model’s superior performance in the DR grading task.