Renal tumor segmentation, visualization, and segmentation confidence using ensembles of neural networks in patients undergoing surgical resection

Abstract Objectives To develop an automatic segmentation model for solid renal tumors on contrast-enhanced CTs and to visualize segmentation with associated confidence to promote clinical applicability. Materials and methods The training dataset included solid renal tumor patients from two tertiary centers undergoing surgical resection and receiving CT in the corticomedullary or nephrogenic contrast media (CM) phase. Manual tumor segmentation was performed on all axial CT slices serving as reference standard for automatic segmentations. Independent testing was performed on the publicly available KiTS 2019 dataset. Ensembles of neural networks (ENN, DeepLabV3) were used for automatic renal tumor segmentation, and their performance was quantified with DICE score. ENN average foreground entropy measured segmentation confidence (binary: successful segmentation with DICE score > 0.8 versus inadequate segmentation ≤ 0.8). Results N = 639/ n = 210 patients were included in the training and independent test dataset. Datasets were comparable regarding age and sex ( p > 0.05), while renal tumors in the training dataset were larger and more frequently benign ( p < 0.01). In the internal test dataset, the ENN model yielded a median DICE score = 0.84 (IQR: 0.62–0.97, corticomedullary) and 0.86 (IQR: 0.77–0.96, nephrogenic CM phase), and the segmentation confidence an AUC = 0.89 (sensitivity = 0.86; specificity = 0.77). In the independent test dataset, the ENN model achieved a median DICE score = 0.84 (IQR: 0.71–0.97, corticomedullary CM phase); and segmentation confidence an accuracy = 0.84 (sensitivity = 0.86 and specificity = 0.81). ENN segmentations were visualized with color-coded voxelwise tumor probabilities and thresholds superimposed on clinical CT images. Conclusions ENN-based renal tumor segmentation robustly performs in external test data and might aid in renal tumor classification and treatment planning. Clinical relevance statement Ensembles of neural networks (ENN) models could automatically segment renal tumors on routine CTs, enabling and standardizing downstream image analyses and treatment planning. Providing confidence measures and segmentation overlays on images can lower the threshold for clinical ENN implementation. Key Points Ensembles of neural networks (ENN) segmentation is visualized by color-coded voxelwise tumor probabilities and thresholds . ENN provided a high segmentation accuracy in internal testing and in an independent external test dataset . ENN models provide measures of segmentation confidence which can robustly discriminate between successful and inadequate segmentations .