Real-time Detection of Bladder Cancer Using Augmented Cystoscopy with Deep Learning: a Pilot Study

Background: Detection of bladder tumors under white light cystoscopy (WLC) is challenging yet impactful on treatment outcomes. Artificial intelligence (AI) holds the potential to improve tumor detection; however, its application in the real-time setting remains unexplored. AI has been applied to previously recorded images for post hoc analysis. In this study, we evaluate the feasibility of real-time AI integration during clinic cystoscopy and transurethral resection of bladder tumor (TURBT) on live, streaming video. Methods: Patients undergoing clinic flexible cystoscopy and TURBT were prospectively enrolled. A real-time alert device system (real-time CystoNet) was developed and integrated with standard cystoscopy towers. Streaming videos were processed in real time to display alert boxes in sync with live cystoscopy. The per-frame diagnostic accuracy was measured. Results and Limitations: Real-time CystoNet was successfully integrated in the operating room during TURBT and clinic cystoscopy in 50 consecutive patients. There were 55 procedures that met the inclusion criteria for analysis including 21 clinic cystoscopies and 34 TURBTs. For clinic cystoscopy, real-time CystoNet achieved per-frame tumor specificity of 98.8% with a median error rate of 3.6% (range: 0 – 47%) frames per cystoscopy. For TURBT, the per-frame tumor sensitivity was 52.9% and the per-frame tumor specificity was 95.4% with an error rate of 16.7% for cases with pathologically confirmed bladder cancers. Conclusions: The current pilot study demonstrates the feasibility of using a real-time AI system (real-time CystoNet) during cystoscopy and TURBT to generate active feedback to the surgeon. Further optimization of CystoNet for real-time cystoscopy dynamics may allow for clinically useful AI-augmented cystoscopy.