MMF-Net: A Novel Multimodal Multiscale Fusion Network for Artery/Vein Segmentation in Retinal Fundus

Automatic artery/vein (Arkers for the early diagnosis of many systemic diseases. Unfortunately, current methods have some limitations in AN segmentation, especially the lack of annotated data and the serious data imbalance. Thus, A novel multimodal multiscale fusion network (MMF-Net) is proposed to alleviate the above problems, which utilizes the internal semantic information of vessels adequately to enhance the AN segmentation. Particularly, the MMF-Net introduces a multimodal (MM) module that could highlight the vessel structure from the original fundus image to constrain the AN image features, which reduces the influence of background noise. In addition, the MMF-Net exploits a multiscale transformation (MT) module to extract the vessel information efficiently from the multimodal feature representations. Finally, A multi-feature fusion (MF) module is applied in MMF-Net to split and reorganize the pixel feature from different scales to improve the robustness of AN segmentation. Experiments on two public benchmark datasets show that our method has achieved superior performance and surpassed other existing state-of-the-art methods in the accuracy of AN segmentation.