The Use of Large Language Models to Generate Education Materials about Uveitis

To assess Large Language Models in generating readable uveitis information and in improving the readability of online health information. Evaluation of Technology. Not applicable. ChatGPT and Bard were asked the following prompts: Prompt A) "considering that the average American reads at a 6th grade level, using the Flesch-Kincaid Grade Level formula, can you write patient-targeted health information on uveitis of around 6th grade level?", and Prompt B) "can you write patient-targeted health information on uveitis that is easy to understand by an average American?". Additionally, ChatGPT and Bard were asked the following prompt from the first-page results of Google when the term "uveitis" was searched: "Considering that the average American reads at a 6th grade level, using the Flesch-Kincaid Grade level formula, can you rewrite the following text to 6th grade level: [insert text]". The readability of each response was analyzed and compared using several metrics described below. The Flesch Kincaid Grade Level (FKGL), is a highly validated readability assessment tool that assigns a grade level to a given text, the total number of words, sentences, syllables, and complex words. Complex words were defined as those with greater than 2 syllables. ChatGPT and Bard generated responses with lower FKGL scores (i.e. easier-to-understand) in response to Prompt A compared to Prompt B. This was only significant for ChatGPT (p<0.0001). The mean FKGL of responses to ChatGPT (6.3 ±1.2) was significantly lower (p<0.0001) than Bard 10.5 ± 0.8. ChatGPT responses also contained less complex words than Bard (p<0.0001). Online health information on uveitis had a mean grade level of 11.0 ± 0.8. ChatGPT lowered the FKGL to 8.0 ± 1.0 (p<0.0001) when asked to re-write the content. Bard was not able to do so (mean FKGL of 11.1 ±1.6). ChatGPT can aid clinicians in producing easier-to-understand health information on uveitis for patients compared to already-existing content. It can also help with reducing the difficulty of the language used for uveitis health information targeted for patients.