Automated segmentation of brain metastases with deep learning: a multi-center, randomized crossover, multi-reader evaluation study

Abstract Background Artificial intelligence has been proposed for brain metastasis (BM) segmentation but it has not been fully clinically validated. The aim of this study was to develop and evaluate a system for BM segmentation. Methods A deep-learning-based BM segmentation system (BMSS) was developed using contrast-enhanced MR images from 488 patients with 10338 brain metastases. A randomized crossover, multi-reader study was then conducted to evaluate the performance of the BMSS for BM segmentation using data prospectively collected from 50 patients with 203 metastases at 5 centers. Five radiology residents and 5 attending radiologists were randomly assigned to contour the same prospective set in assisted and unassisted modes. Aided and unaided Dice similarity coefficients (DSCs) and contouring times per lesion were compared. Results The BMSS alone yielded a median DSC of 0.91 (95% confidence interval, 0.90–0.92) in the multi-center set and showed comparable performance between the internal and external sets (P = .67). With BMSS assistance, the readers increased the median DSC from 0.87 (0.87–0.88) to 0.92 (0.92–0.92) (P < .001) with a median time saving of 42% (40–45%) per lesion. Resident readers showed a greater improvement than attending readers in contouring accuracy (improved median DSC, 0.05 [0.05–0.05] vs 0.03 [0.03–0.03]; P < .001), but a similar time reduction (reduced median time, 44% [40–47%] vs 40% [37–44%]; P = .92) with BMSS assistance. Conclusions The BMSS can be optimally applied to improve the efficiency of brain metastasis delineation in clinical practice.