Deep Learning Based Identification and Segmentation of Lung Tumors on Computed Tomography Images

Rapid and accurate estimation of tumor burden in biomedical images is essential for precisely monitoring cancer progression and assessing therapeutic response. The ability to detect and segment tumors using an automated approach is a key part of this task. Despite recent advances from deep learning, lung tumor delineation remains challenging, particularly when the tumor bounding box is not provided to the model. We hypothesized that clinical radiation oncology contours could supply a large enough dataset of 3D tumor segmentations to enable more accurate models. We developed and validated a deep learning-based model to identify and segment primary and metastatic lung tumors on computed tomography (CT) images.We curated a dataset consisting of CT images and clinical segmentations of 1,916 lung tumors in 1,504 patients who received radiation treatment for one or more primary or metastatic lung tumors. Segmentation quality was independently verified by a radiation oncologist using a custom web application. This dataset was used to train two 3D U-Net convolutional neural networks with varying model properties: one using high-resolution and small input volumes, and one using low-resolution and large input volumes. Models were ensembled together during validation. Performance was evaluated using an external held-out test set of CT images and segmentations from 59 patients with a single primary or metastatic lung tumor, treated at a separate clinical site. This test set consisted of 50 primary lung cancers and 9 metastases. To benchmark model performance against physicians, the test set was also contoured by two additional radiation oncologists.Median tumor volume in the external test set was 80.48 cubic centimeters (interquartile range [IQR]: 14.40 to 177.65). The segmentations generated by the ensembled model produced a mean Dice coefficient of 0.62 (IQR: 0.47 to 0.85) on the test set. The sensitivity for detecting a tumor, as defined by correctly predicting at least one voxel within a ground truth tumor, was 93.2%, and the Dice coefficient for the scans with correctly identified lesions was 0.67 (IQR: 0.53 to 0.85). In comparison, the mean interobserver Dice coefficient for the three physicians on the test set was 0.76 (IQR: 0.70 to 0.84). We observed strong correlation between physician-determined tumor size and model-predicted tumor size (Pearson correlation, r = 0.69, P < 0.0001).An end-to-end deep learning-based model was able to identify and segment lung tumors in a completely automated fashion, with near-expert level performance. Such models could soon be useful for clinical contouring and automatic quantification of tumor burden.