A novel multimodal deep learning model for preoperative prediction of microvascular invasion and outcome in hepatocellular carcinoma

Background Accurate preoperative identification of the microvascular invasion (MVI) can relieve the pressure from personalized treatment adaptation and improve the poor prognosis for hepatocellular carcinoma (HCC). This study aimed to develop and validate a novel multimodal deep learning (DL) model for predicting MVI based on multi-parameter magnetic resonance imaging (MRI) and contrast-enhanced computed tomography (CT). Methods A total of 397 HCC patients underwent both CT and MRI examinations before surgery. We established the radiological models (RCT, RMRI) by support vector machine (SVM), DL models (DLCT_ALL, DLMRI_ALL, DLCT + MRI) by ResNet18. The comprehensive model (CALL) involving multi-modality DL features and clinical and radiological features was constructed using SVM. Model performance was quantified by the area under the receiver operating characteristic curve (AUC) and compared by net reclassification index (NRI) and integrated discrimination improvement (IDI). Results The DLCT + MRI model exhibited superior predicted efficiency over single-modality models, especially over the DLCT_ALL model (AUC: 0.819 vs. 0.742, NRI > 0, IDI > 0). The DLMRI_ALL model improved the performance over the RMRI model (AUC: 0.794 vs. 0.766, NRI > 0, IDI < 0), but no such difference was found between the DLCT_ALL model and RCT model (AUC: 0.742 vs. 0.710, NRI < 0, IDI < 0). Furthermore, both the DLCT + MRI and CALL models revealed the prognostic power in recurrence-free survival stratification (P < 0.001). Conclusion The proposed DLCT + MRI model showed robust capability in predicting MVI and outcomes for HCC. Besides, the identification ability of the multi-modality DL model was better than any single modality, especially for CT.