Attention Based Cross-Domain Synthesis and Segmentation From Unpaired Medical Images

Medical images from different modalities, contrast sequences, and settings can provide diverse information in clinical applications and medical analysis. However, some modalities or contrast sequences may be missing or degraded due to strict timing and artifacts during acquisition, leaving many unpaired data. Therefore, it is meaningful to synthesize realistic medical images with unpaired data. This article proposed a general multi-task method for end-to-end cross-domain synthesis and segmentation network, named SSA-Net, based on cycle generative adversarial network (CycleGAN), using unpaired data for training. A gradient-consistency loss is introduced to supervise the synthesis around the contour, refining the boundaries in synthesized images. And a special shape-consistency term is designed to constrain the anatomical structure in synthesized images, guiding segmentation without target labels. Besides, we introduce the attention mechanism into the generators to focus on some hard-to-learn regions in the images. The FC-DenseNet is employed as a segmentation network to enhance segmentation. Our results demonstrate that the proposed SSA-Net can achieve an S-score of 0.895 on CT images of the CHAOS dataset and a DSC of 0.838 for segmenting the liver, which is a significant increase compared to baseline CycleGAN. Experiment results on four datasets demonstrate the effectiveness of the proposed cross-domain synthesis and segmentation framework.