Prognosis Forecast of Re-Irradiation for Recurrent Nasopharyngeal Carcinoma Based on Deep Learning Multi-Modal Information Fusion

Radiation therapy is the primary treatment for recurrent nasopharyngeal carcinoma. However, it may induce necrosis of the nasopharynx, leading to severe complications such as bleeding and headache. Therefore, forecasting necrosis of the nasopharynx and initiating timely clinical intervention has important implications for reducing complications caused by re-irradiation. This research informs clinical decision-making by making predictions on re-irradiation of recurrent nasopharyngeal carcinoma using deep learning multi-modal information fusion between multi-sequence nuclear magnetic resonance imaging and plan dose. Specifically, we assume that the hidden variables of model data can be divided into two categories: task-consistency and task-inconsistency. The task-consistency variables are characteristic variables contributing to target tasks, while the task-inconsistency variables are not apparently helpful. These modal characteristics are adaptively fused when the relevant tasks are expressed through the construction of supervised classification loss and self-supervised reconstruction loss. The cooperation of supervised classification loss and self-supervised reconstruction loss simultaneously reserves the information of characteristic space and controls potential interference simultaneously. Finally, multi-modal fusion effectively fuses information through an adaptive linking module. We evaluated this method on a multi-center dataset. and found the prediction based on multi-modal features fusion outperformed predictions based on single-modal, partial modal fusion or traditional machine learning methods.