D2FE-GAN: Decoupled dual feature extraction based GAN for MRI image synthesis

Magnetic resonance imaging (MRI) technique can generate various tissue contrasts by using different pulse sequences and parameters. However, obtaining multiple different contrast images for the same patient is sometimes time-consuming and costly. In this paper, we propose a novel generative adversarial network based on decoupled dual feature representations (D2FE-GAN) for cross-modality MRI synthesis. Inspired by the previous works of image style transferring, we argue that the MRI images can be viewed as a compound of underlying information shared among the bodies of modalities (e.g., semantic information), and representative information varying with the styles of modalities (e.g., edges, contrasts). Different from the existing GAN-based methods that pay attention to either the body consistency or the style refinement, the proposed D2FE-GAN method considers both aspects for better synthesis. Specifically, our method decouples the underlying information and the representative information from the source modality and target modality, respectively, through two dissimilar encoders. In response to the invisibility of target modality in testing phase, we propose to employ a Residual Network firstly to generate an intermediate modality as the pseudo target modality. Subsequently, the decoupled two kinds of information will be integrated through a decoder. Here, we introduce the Adaptive Instance Normalization layer, in which the affine parameters are replaced by the mean and standard deviation of the representative information, thus completing the fusion processing of feature space information. Experimental results on BRATS2015 dataset and IXI dataset show that the proposed method outperforms the state-of-the-art image synthesis approaches in both qualitative and quantitative measures.