Multi-Modal Deep Learning Models for Alzheimer's Disease Prediction Using MRI and EHR

Alzheimer's Disease (AD) is an irreversible and progressive neurodegenerative disorder with three stages: cognitively normal (CN), mild cognitive impairment (MCI), and clinical dementia. Progression and stage prediction of dementia plays an important role in prognosis and treatment. In this work, we developed a multi-modal AD progress prediction model that integrates magnetic resonance imaging (MRI) and electronic health record (EHR) to classify patients into three stages: CN, MCI, and AD. We trained deep auto-encoder to extract features from EHR data, and ResNet and 3D U-Net for MRI imaging data. We developed an entropy-based weighted sum classification method to integrate the classification results from each individual modality to generate final prediction. We experimented on Alzheimer's Disease Neuroimaging Initiative (ADNI) data to demonstrate that the multi-modality integration model outperforms single modality models in accuracy, precision, recall, and F1 scores. In addition, our model achieves competitive performance in comparison with other state-of-the-art multi-modality integration methods on AD progression prediction.