Pan-Cancer Prognosis Prediction Using Multimodal Deep Learning

In the age of precision medicine, cancer prognosis assessment from high-dimensional multimodal data requires powerful computational methods. We present an end-to-end multimodal Deep Learning method, named MultiSurv, for automatic patient risk prediction for a large group of 33 cancer types. The method leverages histophatology microscopy slides combined with tabular clinical information and different types of high-throughput sequencing and microarray molecular data. MultiSurv has high predictive performance over all cancer types after training on different combinations of input data modalities and it can handle missing data seamlessly. MultiSurv thus has the potential to integrate the wide variety of available patient data and assist physicians with cancer patient prognosis.