Predicting glaucoma before onset using a large language model chatbot

Purpose To investigate the capability of ChatGPT for forecasting the conversion from ocular hypertension (OHT) to glaucoma based on the Ocular Hypertension Treatment Study (OHTS). Design Retrospective case-control study. Participants A total of 3008 eyes of 1504 subjects from the OHTS were included in the study. Methods We selected demographic, clinical, ocular, optic nerve head, and visual field (VF) parameters one year prior to glaucoma development from the OHTS participants. Subsequently, we developed queries by converting tabular parameters into textual format based on both eyes of all participants. We used the ChatGPT application program interface (API) to automatically perform ChatGPT prompting for all subjects. We then investigated whether ChatGPT can accurately forecast conversion from OHT to glaucoma based on various objective metrics. Main outcome measure Accuracy, area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and weighted F1 score. Results ChatGPT4.0 demonstrated an accuracy of 75%, AUC of 0.67, sensitivity of 56%, specificity of 78%, and weighted F1 score of 0.77 in predicting conversion to glaucoma one year before onset. ChatGPT3.5 provided an accuracy of 61%, AUC of 0.62, sensitivity of 64%, specificity of 59%, and weighted F1 score of 0.63 in predicting conversion to glaucoma one year before onset. Conclusions The performance of ChatGPT4.0 in forecasting development of glaucoma one year before onset was reasonable. The overall performance of ChatGPT4.0 was consistently higher than ChatGPT3.5. Large language models (LLMs) hold great promise for augmenting glaucoma research capabilities and enhancing clinical care. Future efforts in creating ophthalmology specific LLMs that leverage multi-modal data in combination with active learning may lead to more useful integration with clinical practice and deserve further investigations.