Uncertainty-guided man–machine integrated patient-specific quality assurance

Purpose Providing the confidence level (Uncertainty) of prediction results and guiding patient-specific quality assurance (pQA) can enhance the safety of AI (Artificial intelligence)-based automatic pQA models. However, even state-of-the-art automatic pQA models can only predict the gamma passing rate (GPR) and cannot quantify the prediction uncertainty, limiting the safe clinical translation of these models. This study aims to develop an uncertainty-guided man–machine integrated pQA (UgMi-pQA) method to address this issue. Methods An uncertainty-aware dual-task deep learning (UDDL) model, combined with an interwoven training method and Monte Carlo dropout approximation Bayesian inference, to enable simultaneous output of the predicted GPR and corresponding total prediction uncertainty to guide pQA. 1541 pairs of field fluences and GPRs collected from 165 glioma, 50 lung (conventional fractionation), and 20 liver cases were separated for the UDDL model training, validation, calibration, and test in a ratio of 7:1:1:1, respectively. Furthermore, 413 pairs of fluences and GPRs collected from 12 breast, 10 cervix, 9 esophagus, 8 tongue, and 12 lung SBRT cases were gathered for the out-of-distribution (OOD) detection. Results Clinical accuracy of 100.0% was reached with only 61.7% of the workload. Samples with substantial prediction errors and failed samples with low label GPR (<95%) could be successfully screened out. The capability ranges of two different models were both successfully identified with the prediction uncertainty significantly larger for OOD samples than for in-distribution samples (p < 0.01). Conclusion This study presents the first work on uncertainty quantification for deep learning automatic pQA tasks. The UgMi-pQA method can balance the efficiency and safety of the automatic pQA models and promote their clinical application.