Quality Assurance based on Deep Learning for Pelvic OARs Delineation in Radiotherapy

Correct delineation of organs at risk (OARs) is an important step for radiotherapy and it is also a time-consuming process that depends on many factors.An automatic quality assurance (QA) method based on deep learning (DL) was proposed to improve efficiency for detecting contouring errors of OARs.A total of 180 planning CT scan sets at the pelvic site and the corresponding OARs contours from clinics were enrolled in this study. Among them, 140 cases were randomly chosen as the training datasets, 20 cases were used as the validation datasets, and the remaining 20 cases were used as the test datasets. DL-based models were trained through data curation for data cleaning based on the Dice similarity coefficient and the 95th percentile Hausdorff distance between the original contours and the predicted contours. All contouring errors could be classified into two types; minor modification required and major modification required. The pass criteria were established using Bias- Corrected and Accelerated bootstrap on 20 manually reviewed validation datasets. The performance of the QA method was evaluated with the metrics of sensitivity, specificity, the area under the receiving operator characteristic curve (AUC), and detection rate sensitivity on the 20 test datasets.For all OARs, segmentation results after data curation were superior to those without. The sensitivity of the QA method was greater than 0.890 and the specificity was higher than 0.975. The AUCs were 0.948, 0.966, 0.965, and 0.932 for the bladder, right femoral head, left femoral head, and rectum, respectively. Almost all major errors could be detected by the automatic QA method, and the lowest detection rate sensitivity of minor errors was 0.863 for the rectum.QA of OARs is an important step for the correct implementation of radiotherapy. The DL-based QA method proposed in this study showed a high potential to automatically detect contouring errors with high precision. The method can be integrated into the existing radiotherapy procedures to improve the efficiency of delineating the OARs.