Evaluation of an artificial intelligence algorithm for assisting the Paris System in reporting urinary cytology: A pilot study

Background The Paris System for Reporting Urinary Cytology (TPS) has been shown to improve bladder cancer diagnosis. Advances in artificial intelligence (AI) may assist and improve the clinical workflow by applying TPS in routine diagnostic services. Methods A deep‐learning–based algorithm was developed to identify urothelial cancer candidate cells using whole‐slide images (WSIs). In the testing cohort, 131 urine cytology slides were retrospectively retrieved and analyzed using this AI algorithm. The authors compared the performance of one cytopathologist and two cytotechnologists using AI‐assisted digital urine cytology. Then, the AI‐assisted WSIs were evaluated in the clinical workflow. The cytopathologist first made a diagnosis by reviewing the AI‐inferred WSIs and quantitative data (nuclear‐to‐cytoplasmic ratio and nuclear size) for each sample. After a washout period, the same cytopathologist made a diagnosis for the same samples using direct microscopy. All diagnosis results were compared with the expert panel consensus. Results The AI‐assisted diagnosis by the two cytotechnologists and the one cytopathologist demonstrated performance results that were comparable to the expert panel consensus (sensitivity, 79.5% and 82.1% vs. 92.3%, respectively; specificity, 100% and 98.9% vs. 100%, respectively). Furthermore, the performance of the AI‐assisted WSIs compared with the microscopic diagnosis by the cytopathologist demonstrated superior sensitivity (92.3% vs. 87.2%) and negative predictive value (96.8% vs. 94.8%). In addition, the AI‐assisted reporting demonstrated near perfect agreement with the expert panel consensus (κ = 0.944) and the microscopic diagnosis (κ = 0.862). Conclusions The AI algorithm developed by the authors effectively assisted TPS‐based reporting by providing AI‐inferred WSIs and quantitative data.