Real-Time Detection of Ureteral Orifice in Urinary Endoscopy Videos Based on Deep Learning

In urology endoscopic procedures, the Ureteral Orifice (UO) finding is crucial but may be challenging for inexperienced doctors. Generally, it is difficult to identify UOs intraoperatively due to the presence of a large median lobe, obstructing tumor, previous surgery, etc. To automatically identify various types of UOs in the video, we propose a real-time deep learning system in UO identification and localization in urinary endoscopy videos, and it can be applied to different types of urinary endoscopes. Our UO detection system is mainly based on Single Shot MultiBox Detector (SSD), which is one of the state-of-the-art deep-learning based detection networks in natural image domain. For the preprocessing, we apply both general and specific data augmentation strategies which have significantly improved all evaluation metrics. For the training steps, we only utilize rescetoscopy images which have more complex background information, and then, we use ureteroscopy images for testing. Simultaneously, we demonstrate that the model trained with rescetoscopy images can be successfully applied in the other type of urinary endoscopy images with four evaluation metrics (precision, recall, F1 and F2 scores) greater than 0.8. We further evaluate our model based on four independent video datasets which comprise both rescectoscopy videos and ureteroscopy videos. Extensive experiments on the four video datasets demonstrate that our deep-learning based UO detection system can identify and locate UOs of two different urinary endoscopes in real time with average processing time equal to 25 ms per frame and simultaneously achieve satisfactory recall and specificity.