Deep learning for diagnosis and survival prediction in soft tissue sarcoma

•A DLM was able to classify five of the most common STS subtypes from histology alone.•When aided by the DLM, pathologists were more accurate, faster, and more certain in their diagnosis.•A similar DLM was able to predict the disease-specific survival status in the most common STS subtype.•The DLM's prediction was an independent prognostic factor.•New image features associated with survival could be identified. BackgroundClinical management of soft tissue sarcoma (STS) is particularly challenging. Here, we used digital pathology and deep learning (DL) for diagnosis and prognosis prediction of STS.Patients and methodsOur retrospective, multicenter study included a total of 506 histopathological slides from 291 patients with STS. The Cancer Genome Atlas cohort (240 patients) served as training and validation set. A second, multicenter cohort (51 patients) served as an additional test set. The use of the DL model (DLM) as a clinical decision support system was evaluated by nine pathologists with different levels of expertise. For prognosis prediction, 139 slides from 85 patients with leiomyosarcoma (LMS) were used. Area under the receiver operating characteristic (AUROC) and accuracy served as main outcome measures.ResultsThe DLM achieved a mean AUROC of 0.97 (±0.01) and an accuracy of 79.9% (±6.1%) in diagnosing the five most common STS subtypes. The DLM significantly improved the accuracy of the pathologists from 46.3% (±15.5%) to 87.1% (±11.1%). Furthermore, they were significantly faster and more certain in their diagnosis. In LMS, the mean AUROC in predicting the disease-specific survival status was 0.91 (±0.1) and the accuracy was 88.9% (±9.9%). Cox regression showed the DLM’s prediction to be a significant independent prognostic factor (P = 0.008, hazard ratio 5.5, 95% confidence interval 1.56-19.7) in these patients, outperforming other risk factors.ConclusionsDL can be used to accurately diagnose frequent subtypes of STS from conventional histopathological slides. It might be used for prognosis prediction in LMS, the most prevalent STS subtype in our cohort. It can also help pathologists to make faster and more accurate diagnoses. This could substantially improve the clinical management of STS patients. Clinical management of soft tissue sarcoma (STS) is particularly challenging. Here, we used digital pathology and deep learning (DL) for diagnosis and prognosis prediction of STS. Our retrospective, multicenter study included a total of 506 histopathological slides from 291 patients with STS. The Cancer Genome Atlas cohort (240 patients) served as training and validation set. A second, multicenter cohort (51 patients) served as an additional test set. The use of the DL model (DLM) as a clinical decision support system was evaluated by nine pathologists with different levels of expertise. For prognosis prediction, 139 slides from 85 patients with leiomyosarcoma (LMS) were used. Area under the receiver operating characteristic (AUROC) and accuracy served as main outcome measures. The DLM achieved a mean AUROC of 0.97 (±0.01) and an accuracy of 79.9% (±6.1%) in diagnosing the five most common STS subtypes. The DLM significantly improved the accuracy of the pathologists from 46.3% (±15.5%) to 87.1% (±11.1%). Furthermore, they were significantly faster and more certain in their diagnosis. In LMS, the mean AUROC in predicting the disease-specific survival status was 0.91 (±0.1) and the accuracy was 88.9% (±9.9%). Cox regression showed the DLM’s prediction to be a significant independent prognostic factor (P = 0.008, hazard ratio 5.5, 95% confidence interval 1.56-19.7) in these patients, outperforming other risk factors. DL can be used to accurately diagnose frequent subtypes of STS from conventional histopathological slides. It might be used for prognosis prediction in LMS, the most prevalent STS subtype in our cohort. It can also help pathologists to make faster and more accurate diagnoses. This could substantially improve the clinical management of STS patients.