RCAR-UNet: Retinal vessel segmentation network algorithm via novel rough attention mechanism

The health status of the retinal blood vessels is a significant reference for rapid and non-invasive diagnosis of various ophthalmological, diabetic, and cardio-cerebrovascular diseases. However, retinal vessels are characterized by ambiguous boundaries, with multiple thicknesses and obscured lesion areas. These phenomena cause deep neural networks to face the characteristic channel uncertainty when segmenting retinal blood vessels. The uncertainty in feature channels will affect the channel attention coefficient, making the deep neural network incapable of paying attention to the detailed features of retinal vessels. This study proposes a retinal vessel segmentation via a rough channel attention mechanism. First, the method integrates deep neural networks to learn complex features and rough sets to handle uncertainty for designing rough neurons. Second, a rough channel attention mechanism module is constructed based on rough neurons, and embedded in U-Net skip connection for the integration of high-level and low-level features. Then, the residual connections are added to transmit low-level features to high-level to enrich network feature extraction and help back-propagate the gradient when training the model. Finally, multiple comparison experiments were carried out on three public fundus retinal image datasets to verify the validity of Rough Channel Attention Residual U-Net (RCAR-UNet) model. The results show that the RCAR-UNet model offers high superiority in accuracy, sensitivity, F1, and Jaccard similarity, especially for the precise segmentation of fragile blood vessels, guaranteeing blood vessels’ continuity.