Validation of an automated machine learning algorithm for the detection and analysis of cerebral aneurysms

OBJECTIVE Machine learning algorithms have shown groundbreaking results in neuroimaging. The authors herein evaluated the performance of a newly developed convolutional neural network (CNN) to detect and analyze intracranial aneurysms (IAs) on CTA. METHODS Consecutive patients with CTA studies between January 2015 and July 2021 at a single center were identified. The ground truth determination of cerebral aneurysm presence or absence was made from the neuroradiology report. The primary outcome was the performance of the CNN in detecting IAs in an external validation set, measured using area under the receiver operating characteristic curve statistics. Secondary outcomes included accuracy for location and size measurement. RESULTS The independent validation imaging data set consisted of 400 patients with CTA studies, median age 40 years (IQR 34 years) and 141 (35.3%) of whom were male; 193 patients (48.3%) had a diagnosis of IA on neuroradiologist evaluation. The median maximum IA diameter was 3.7 mm (IQR 2.5 mm). In the independent validation imaging data set, the CNN performed well with 93.8% sensitivity (95% CI 0.87–0.98), 94.2% specificity (95% CI 0.90–0.97), and a positive predictive value of 88.2% (95% CI 0.80–0.94) in the subgroup with an IA diameter ≥ 4 mm. CONCLUSIONS The described Viz.ai Aneurysm CNN performed well in identifying the presence or absence of IAs in an independent validation imaging set. Further studies are necessary to investigate the impact of the software on detection rates in a real-world setting.