Multi-task Multi-instance Learning for Jointly Diagnosis and Prognosis of Early-Stage Breast Invasive Carcinoma from Whole-Slide Pathological Images

With the tremendous progress brought by artificial intelligence, many whole-slide pathological images (WSIs) based machine learning models are designed to estimate the clinical outcome of human cancers. However, most of the existing studies treat the prognosis and diagnosis tasks separately, which overlooks the fact that the diagnosis information indicating the severity of the disease that is highly related to the patients' survival. In addition, it is still challenge to design machine learning models to analyze WSIs since a WSI is of large size but only annotate with coarse labels, it increasingly becomes a research hotspot for the development of automated WSI analysis tools in a scenario without fully annotated data. Based on the above considerations, we propose a multi-task multi-instance (WSI-MTMI) learning method that can simultaneously conduct the prognosis and diagnosis tasks from WSIs. Specifically, inspired by the fact that taking the associations between diagnosis and prognosis tasks can improve the generalization ability for each individual task, we firstly adopt multi-instance learning algorithms to aggregate the patches derived from WSIs by considering both the common and specific task information. Then, we design a novel cross-attention network that can effectively identify useful information shared across different tasks. To evaluate the effectiveness of the proposed method, we test it on the early-stage breast invasive carcinoma (BRCA) derived from the Cancer Genome Atlas project (TCGA), and the experimental results indicate that our method can achieve better performance on both diagnosis and prognosis tasks than the related methods.