Dual-stream Representation Fusion Learning for accurate medical image segmentation

Accurate segmenting regions of interest in various medical images are essential to clinical research and applications. Although deep learning-based methods have achieved good results, the fully automated segmentation results still need to be refined on the tininess, complexities, and irregularities of lesion shapes. To address this issue, we propose a Dual-stream Representation Fusion Learning (DRFL) paradigm for accurate clinical segmentation, including Dual-stream Fusion Module, Representation Fusion Transformer Module and Peakiness Fusion Attention Module. Specifically, Dual-stream Fusion Module can simultaneously generate binary masks and high-resolution images with segmentation stream and super-resolution stream that share a feature extractor, then both prediction outputs are merged as the input of Fusion Module to further improve the performance of the network for generating the final segmentation result; Representation Fusion Transformer Module is lightweight to fuse high-resolution representation and fine-grained structure representation; Peakiness Fusion Attention Module can capture more salient features while fusing more spatial information to improve the performance of the network. The effectiveness of our dual-stream representation fusion learning is validated on different medical image segmentation tasks, and extensive experiments show that our DRFL outperforms the state-of-the-art methods in segmentation quality of lung nodule segmentation, lung segmentation, cell contour segmentation, and prostate segmentation. Our code is available at https://github.com/Rongtao-Xu/RepresentationLearning/tree/main/DRFL-EAAI2023.