Collaborative Image Synthesis and Disease Diagnosis for Classification of Neurodegenerative Disorders with Incomplete Multi-modal Neuroimages

The missing data issue is a common problem in multi-modal neuroimage (e.g., MRI and PET) based diagnosis of neurodegenerative disorders. Although various generative adversarial networks (GANs) have been developed to impute the missing data, most current solutions treat the image imputation and disease diagnosis as two standalone tasks without considering the impact of diagnosis on image synthesis, leading to less competent synthetic images to the diagnosis task. In this paper, we propose the collaborative diagnosis-synthesis framework (CDSF) for joint missing neuroimage imputation and multi-modal diagnosis of neurodegenerative disorders. Under the CDSF framework, there is an image synthesis module (ISM) and a multi-modal diagnosis module (MDM), which are trained in a collaborative manner. Specifically, ISM is trained under the supervision of MDM, which poses the feature-consistent constraint to the cross-modality image synthesis, while MDM learns the disease-related multi-modal information from both real and synthetic multi-modal neuroimages. We evaluated our CDSF model against five image synthesis methods and three multi-modal diagnosis models on an ADNI datasets with 1464 subjects. Our results suggest that the proposed CDSF model not only generates neuroimages with higher quality, but also achieves the state-of-the-art performance in AD identification and MCI-to-AD conversion prediction.