MRI and RNA-seq fusion for prediction of pathological response to neoadjuvant chemotherapy in breast cancer

Accurate prediction of the pathological complete response (pCR) to neoadjuvant chemotherapy (NAC) is crucial for precise treatment of breast cancer. However, current studies mainly rely on single-modal data, with limited studies focusing on multimodal data. In this study, we developed and validated a deep learning-based multimodal fusion model that predicts the response of breast tumor to NAC by integrating multi-parametric magnetic resonance imaging (MRI) and RNA sequencing (RNA-seq) information related to breast tumor. For comparison, we separately built four single-modal models with either MR images or RNA-seq data. Moreover, our approach has demonstrated better performance in integrating MR images and RNA-seq data. The average accuracy is 90.20% and area under the ROC curve(AUC) is 0.936 for our model. These findings indicate that our proposed approach has achieved higher accuracy in predicting the pathological response to NAC.