Comparison of ChatGPT and Traditional Patient Education Materials for Men’s Health

You have accessUrology PracticeHealth Policy1 Jan 2024Comparison of ChatGPT and Traditional Patient Education Materials for Men’s HealthThis article is commented on by the following:Editorial CommentEditorial Comment Yash B. Shah, Anushka Ghosh, Aaron R. Hochberg, Eli Rapoport, Costas D. Lallas, Mihir S. Shah, and Seth D. Cohen Yash B. ShahYash B. Shah https://orcid.org/0000-0003-1551-7611 Department of Urology, Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, Pennsylvania , Anushka GhoshAnushka Ghosh Department of Urology, Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, Pennsylvania , Aaron R. HochbergAaron R. Hochberg Department of Urology, Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, Pennsylvania , Eli RapoportEli Rapoport Department of Urology, NYU Langone, New York, New York , Costas D. LallasCostas D. Lallas Department of Urology, Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, Pennsylvania , Mihir S. ShahMihir S. Shah Department of Urology, Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, Pennsylvania , and Seth D. CohenSeth D. Cohen *Corresponding Author: Seth D. Cohen, MD, MPH, Department of Urology, NYU Grossman School of Medicine Director, Sexual Dysfunction Program, 222 E 41st St, 11th Floor, New York, NY 10017 ( E-mail Address: [email protected] Department of Urology, NYU Langone, New York, New York View All Author Informationhttps://doi.org/10.1097/UPJ.0000000000000490AboutAbstractPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookTwitterLinked InEmail Abstract Introduction: ChatGPT is an artificial intelligence platform available to patients seeking medical advice. Traditionally, urology patients consulted official provider-created materials, particularly the Urology Care Foundation™ (UCF). Today, men increasingly go online due to the rising costs of health care and the stigma surrounding sexual health. Online health information is largely inaccessible to laypersons as it exceeds the recommended American sixth to eighth grade reading level. We conducted a comparative assessment of patient education materials generated by ChatGPT vs UCF regarding men’s health conditions. Methods: All 6 UCF men’s health resources were identified. ChatGPT responses were generated using patient questions obtained from UCF. Adjusted ChatGPT responses were generated by prompting, “Explain it to me like I am in sixth grade.” Textual analysis was performed using sentence, word, syllable, and complex word count. Six validated formulae were used for readability analysis. Two physicians independently scored responses for accuracy, comprehensiveness, and understandability. Statistical analysis involved Wilcoxon matched-pairs test. Results: ChatGPT responses were longer and more complex. Both UCF and ChatGPT failed official readability standards, although ChatGPT performed significantly worse across all 6 topics (all P < .001). Conversely, adjusted ChatGPT readability typically surpassed UCF, even meeting the recommended level for 2 topics. Qualitatively, UCF and ChatGPT had comparable accuracy, although ChatGPT had better comprehensiveness and worse understandability. Conclusions: When comparing readability, ChatGPT-generated education is less accessible than provider-written content, although neither meets the recommended level. Our analysis indicates that specific artificial intelligence prompts can simplify educational materials to meet national standards and accommodate individual literacy. Patients increasingly go online for health advice, particularly for sensitive issues like sexual health.1,2 As technology companies continue to innovate artificial intelligence (AI) applications, this trend will increase.3-5 Modern large language models, namely OpenAI’s ChatGPT, allow free text entry and generate novel responses.6 Of note, these responses are often long and convoluted. It is unclear whether AI carries downsides similar to social media, where men’s health research has shown misinformation and poor quality.2,7,8 ChatGPT can perform diverse physician-oriented tasks, including writing notes, improving empathy in written communications,1,9 completing certification exams,10 and simplifying scientific literature.11 However, there is limited understanding of AI use by patients, who have recently begun turning to new platforms for their health information. Misinformation on ChatGPT is apparently minimal but real.5,6,12-14 Particularly worrisome is AI hallucination, where misinformation may arise from subjective, emotion-laden questioning.3,4 Equally as important is understandability. It remains unknown whether ChatGPT generates responses at levels appropriate for laypeople. The National Institutes of Health and American Medical Association recommend education at the sixth to eighth grade level.11,15,16 Unfortunately, most existing online resources fall above this standard and are inaccessible to average patients.17 A recent urology study found that ChatGPT cannot currently self-evaluate readability for written content.18 Education regarding men’s health was traditionally obtained from provider-created materials, particularly the Urology Care Foundation™ (UCF), which is the primary patient-facing service of the AUA.4,19 As AI becomes increasingly popular amongst patients, we must understand its strengths and limitations. Here, we aimed to evaluate the utility of ChatGPT in improving accessibility and quality of educational information within men’s health, comparing responses with gold-standard UCF resources. Methods Data Collection All 6 UCF men’s health articles (erectile dysfunction, premature ejaculation, low testosterone, sperm retrieval, penile augmentation, and male infertility) were identified. Article content included definition, etiology, diagnostics, treatment, prognosis, and common concerns. UCF subheadings were used verbatim, or if necessary, converted into question form, to create 60 patient questions (Supplemental Table 1, https://www.urologypracticejournal.com). ChatGPT3.5 was used on July 20, 2023, to generate responses within unique chatbot instances. To elicit adjusted ChatGPT (ChatGPT-a) responses at the minimum appropriate reading level, as defined by the National Institutes of Health and American Medical Association, the prompt “Explain it to me like I am in sixth grade” was added. Readability Descriptive analysis evaluated textual composition, including sentence, word, complex word, and syllable count. Readability calculation (Table 1) utilized the following validated formulae: Flesch-Kincaid Reading Ease Score, Flesch-Kincaid Grade Level, Gunning-Fog Score, Simple Measure of Gobbledygook, Coleman-Liau Index, and Automated Readability Index.1,20,21 Table 1. Calculation and Interpretation of 6 Validated Readability Metrics Readability metric Formula Scoring Flesch-Kincaid Reading Ease Score 206.835 − 1.015(words/sentences) – 84.6(syllables/words) Range: 0-100 Interpretation: higher score indicates better readability Recommended: ≥70 Flesch-Kincaid Grade Level 0.39(words/sentences) + 11.8(syllables/words) – 15.59 Range: American school grade level Interpretation: lower score indicates better readability Recommended: ≤8 Gunning-Fog Index 0.4([words/sentences] + 100[x/words]) x = polysyllabic words excluding common jargon, compound words, suffixes, or proper nouns Simple Measure of Gobbledygook 1.043(x[30/sentences])1/2 + 3.129 x = polysyllabic words Coleman-Liau Index (0.0588x – 0.296y) – 15.8 x = mean letters per 100 words y = mean sentences per 100 words Automated Readability Index 4.71([characters/words] + 0.5[words/sentences]) Quality Two blinded urologists scored responses for accuracy (“Is the response evidence-based and medically accurate?”), comprehensiveness ("Does the response provide sufficient information to fully inform patients about their diagnosis/treatment?”), and understandability (“Can the response be easily understood by average patients?”) using Likert items (1 = Poor, 2 = Needs Improvement, 3 = Fair, 4 = Good, 5 = Excellent).1,20-24 Accuracy was assessed using AUA guidelines. Comprehensiveness and understandability were scored qualitatively. Statistical Analysis To assess differences in readability or quality between UCF and ChatGPT responses, paired analyses were conducted for each prompt using Wilcoxon signed-rank tests. Statistical analyses were conducted using GraphPad Prism V9.0.2. Alpha was 0.05. Results Representative responses to 1 question are presented in Supplemental Table 2 (https://www.urologypracticejournal.com). ChatGPT provided significantly longer responses with a greater proportion of complex words when compared to UCF (Figure 1). ChatGPT had greater volume of information, represented by sentence (17.3 vs 16.1, P = .030) and word count (329.3 vs 230.8, P < .001; Table 2). Complex word usage and sentence length were higher on ChatGPT for all topics (P < .001). ChatGPT-a demonstrated comparable length to ChatGPT but significantly lower complex word usage (16.0% vs 30.0%, P < .001). Figure 1. Descriptive analysis of text given in response to individual patient questions from ChatGPT vs Urology Care Foundation™ (UCF). Box-and-whisker plot represents the median and interquartile range while points represent statistical outliers. Metrics of interest included sentence count (A), word count (B), complex word count as defined by polysyllabic words but excluding common jargon or proper nouns (C), percentage of complex words in the passage (D), average number of words per sentence (E), and average number of syllables per word (F). Table 2. Descriptive Analysis of Text From Urology Care Foundation™ vs ChatGPT Sentence count Word count Complex word count % Complex words Average words per sentence Average syllables per word UCF ChatGPT UCF ChatGPT UCF ChatGPT UCF ChatGPT UCF ChatGPT UCF ChatGPT Erectile dysfunctiona 12.1 18.0 195.8 333.4 24.6 97.8 13.0 29.3 17.4 18.7 1.5 1.9 Premature ejaculationa 21.2 15.8 283.7 293.3 37.8 97.2 13.0 33.4 13.3 18.6 1.5 2.1 Low testosteronea 16.4 15.8 267.4 289.3 36.6 83.6 14.0 29.0 20.3 20.3 1.5 2.0 Sperm retrievala 15.1 17.6 202.8 360.3 33.9 104.0 14.7 28.8 16.8 20.8 1.5 1.9 Penile augmentationa 14.0 7.0 149.0 144.0 25.0 34.0 16.8 23.6 10.6 20.6 1.7 1.9 Male infertilitya 18.3 18.4 239.0 358.7 40.2 111.7 14.2 31.1 14.7 19.7 1.6 2.0 Mean ± SDb 16.1 ± 20.4 17.3 ± 3.9 230.8 ± 270.5 329.3 ± 56.8 34.0 ± 49.4 99.1 ± 22.4 13.8 ± 5.6 30.0 ± 4.3 16.5 ± 7.7 19.6 ± 4.0 1.5 ± 0.14 2.0 ± 0.12 P value .030 < .001 < .001 < .001 < .001 < .001 Abbreviations: SD, standard deviation; UCF, Urology Care Foundation™. Represents mean for all patient questions within men’s health topic. Represents mean and standard deviation across all 60 patient questions without grouping by topic. On readability analysis, both ChatGPT and UCF exceeded the recommended sixth to eighth grade level across all topics and formulae (Supplemental Table 3, https://www.urologypracticejournal.com). ChatGPT had poorer readability than UCF across all 6 formulae (all P < .001; Figure 2). Figure 2. Distribution of readability scores for responses to individual patient questions from ChatGPT vs Urology Care Foundation™ (UCF), stratified by readability formula. Box-and-whisker plot represents the median and interquartile range while points represent statistical outliers. Of note, a higher score demonstrates improved readability on Flesch-Kincaid Reading Ease Score (FKRE; A), while a lower score demonstrates improved readability on Flesch-Kincaid Grade Level (FKGL; B), Gunning-Fog Score (GFS; C), Simple Measure of Gobbledygook (SMOG; D), Coleman-Liau Index (CLI; E), and Automated Readability Index (ARI; F). UCF demonstrated better readability (P < .001) across all 6 formulae. However, adjustment for user literacy (ChatGPT-a) demonstrated notable improvement for all men’s health conditions, even surpassing UCF for 4/6 topics (Supplemental Figure 1, https://www.urologypracticejournal.com). Overall, 2 topics (low testosterone and sperm retrieval) fell within the sixth to eighth grade recommendation, while another 2 (erectile dysfunction and male infertility) met the eighth-grade level (Figure 3). Figure 3. Required education level for adequate comprehension of ChatGPT, Urology Care Foundation™ (UCF), and adjusted ChatGPT (ChatGPT-a) patient materials, stratified by men’s health topic. Dashed line indicates the upper limit of the recommended sixth to eighth grade reading level per the National Institutes of Health and American Medical Association. Points represent the distribution of required grade level for understanding responses to individual patient questions. Bar represents the mean across all patient questions within 1 men’s health topic. Quality analysis demonstrated that ChatGPT-a content closely paralleled ChatGPT. When mean scores were analyzed across reviewers, accuracy was comparable for ChatGPT and UCF. Conversely, ChatGPT was more comprehensive than UCF (P < .001), indicating potential value to its lengthiness. Nonetheless, understandability scored lower with ChatGPT (P = .020), supporting our quantitative findings (Table 3). Table 3. Qualitative Physician Rating of Accuracy, Comprehensiveness, and Understandability for Patient Educational Material From ChatGPT vs Urology Care Foundation™ ChatGPT vs UCF Accuracy Comprehensiveness Understandability Platform ChatGPT UCF ChatGPT UCF ChatGPT UCF Overall mean ± SD 4.70 ± 0.64 4.69 ± 0.59 4.54 ± 0.74 4.03 ± 1.04 4.59 ± 0.65 4.88 ± 0.32 P value .810 < .001 .020 Abbreviations: SD, standard deviation; UCF, Urology Care Foundation™. Discussion Our study demonstrates poor readability for online men’s health resources. UCF, one of the most popular patient platforms, did not meet the recommended level, while ChatGPT performed even worse. These findings were supported by 6 validated metrics and persisted across men’s health topics. Patients with reduced health literacy, facing interplay with social and racial inequity, experience poorer treatment adherence and satisfaction with care.25 Hence, materials following national readability guidelines can benefit outcomes and overall public health.11 Searching PubMed for patient-oriented ChatGPT research, we identified only 8 urology publications, and none within sexual medicine specifically. Given the lack of literature, it is difficult for urologists to appreciate information which patients may encounter outside the controlled clinic setting. Nearly 3/4 Americans go online for medical advice and admit influence on their decision-making.26,27 Suboptimal information can push patients towards ill-advised management options or reduced adherence to guideline-driven advice. Social media within men’s health has poor quality and gaps in comprehensiveness, as we previously reported.2,7 It is reasonable to question whether AI carries parallel risks. When providers seek ChatGPT’s help, responses follow evidence-based recommendations and practice guidelines.28 However, when Cocci et al inputted 100 urology case studies into ChatGPT, similar to how a provider consults resources to develop differential diagnoses for patients, responses were at the 15.8 grade level.1 It is unclear whether physician-oriented studies can be translated to patients seeking help on the platform. A prostate cancer study compared ChatGPT with European Association of Urology materials, finding ChatGPT underperformed in quality and accuracy.29 Regarding readability, Davis et al obtained 18 popular urology patient questions from Google Trends, finding ChatGPT responded around the 13.5 grade level.21 A Society of Interventional Radiology study endorsed similar conclusions; ChatGPT met 12th grade while official materials met 11th grade levels.3 Although our study supports evidence from other specialties that ChatGPT is difficult to read for average patients, we found notable improvement when prompting AI accommodation to a particular literacy level. ChatGPT-a improved readability universally, enough to meet national standards for 2 topics. Of note, minimal improvement in the ChatGPT-a penile augmentation response may be attributable to low sample size as only 1 patient question was available. Importantly, 2 practicing urologists scored ChatGPT response accuracy similarly to physician-led UCF materials, suggesting promise in AI-created education. Notably, the scorers found that understandability was qualitatively poorer for ChatGPT, confirming our quantitative results. Previous data indicate physicians are unaware of layperson literacy given their own education; often patients seek third-party materials because education provided in the clinic is inaccessible.30 Although patients would ideally access materials written by providers, these lack readability; quantitative evaluation remains essential in ensuring trusted online materials are accessible. Research can allow organizations including the AUA to offer national guidance assisting clinicians in appreciating these emerging technologies. This would ensure that clinicians use evidence-based science communication strategies to produce educational material that caters to their clinic’s needs. To date, physician-led resources like UCF have not met readability recommendations despite calls to action. Future processes for developing patient-facing materials may involve physicians writing resources and subsequently using AI to simplify their language. Similarly, physicians and health care organizations may use AI to generate wholly new materials which can be subsequently proofread by staff. This would allow rapid generation of patient-specific educational content. Otherwise, patients may use AI to reword existing complicated resources. Further research may determine the ChatGPT-a prompt which best accommodates individual literacy, going beyond the simple, “Explain it to me like I am in sixth grade.” Therefore, ChatGPT can encourage personalized education and reduce time urologists spend responding to online patient portal messages, simultaneously accomplishing scale and democratization of information.31 The current ChatGPT version is not specialized in medical information and was trained using the entire Internet without controls for validity.12,31 Moreover, it was only trained using data through 2021, although some studies suggest the model is self-learning from newer 2022 to 2023 data.5 This creates risks of inaccuracy and limited ability to incorporate emerging research.5,14,28 As total medical knowledge expands exponentially, it is impossible for physicians to serve as encyclopedias; AI can assist physicians with this challenge.30,32,33 OpenAI leadership is planning future iterations aimed at specialized industries, having already formed a collaboration with Epic. A patient-oriented version trained using only validated medical literature and designed to return short, accessible advice could serve a triage role. Integration into the electronic medical record or patient portal would be valuable. With development, this model may self-check responses by incorporating quantitative quality and readability formulae.18 Partnerships between expert stakeholders, the AUA, public health agencies, and AI companies can create improved patient-oriented tools.20,30 Although this study offers early insights into AI within urology, there are limitations. Namely, ChatGPT’s stochasticity entails differences in results across points in time. We asked 60 questions across 6 topics to reduce stochasticity implications.4,5,14 Our methodology only included 1 iteration per question and was completed on a single date to reduce confounding. We focused on UCF as it is utilized nationally, widely trusted, and includes several sexual health conditions. However, UCF may not accurately represent the quality of all physician-created online resources. Additionally, readability formulae cannot score multimedia, such as videos or images, which may influence understanding. Finally, accuracy, quality, and comprehensiveness were subjectively evaluated by 2 urologists, carrying risk of bias. However, this methodology has been previously published and carries intrinsic value exhibiting physicians’ ability to reflect when writing educational materials for patients.1,20-24 Future studies may involve real patients scoring educational materials for utility and accessibility. Conclusions Our study indicates low readability within official men’s health patient materials. Readability for patient-oriented ChatGPT content was even worse, although prompting to increase accessibility in a patient-specific manner notably improved responses. Accuracy was similar for both provider and AI-supported resources. Refined AI models can play an integral role in patient education within men’s health and beyond. References 1. . Quality of information and appropriateness of ChatGPT outputs for urology patients. Prostate Cancer Prostatic Dis. 2023; 10.1038/s41391-023-00705-y. Crossref, Google Scholar 2. . Analysis of patient education materials on TikTok for erectile dysfunction treatment. Int J Impot Res. 2023; 10.1038/s41443-023-00726-0. Crossref, Google Scholar 3. . Evaluation of an artificial intelligence chatbot for delivery of IR patient education material: a comparison with societal website content. J Vasc Interv Radiol. 2023; 34(10):1760.e32-1768.e32. Crossref, Google Scholar 4. . Potential safety concerns in use of the artificial intelligence chatbot ‘ChatGPT' for perioperative patient communication. BJU Int. 2023; 132(2):157-159. Crossref, Medline, Google Scholar 5. . Potential and pitfalls of ChatGPT and natural-language artificial intelligence models for diabetes education. Diabetes Care. 2023; 46(5):e103-e105. Crossref, Medline, Google Scholar 6. . Can the ChatGPT and other large language models with internet-connected database solve the questions and concerns of patient with prostate cancer and help democratize medical knowledge?. J Transl Med. 2023; 21(1):269. Crossref, Medline, Google Scholar 7. . TikTok as a source of information regarding premature ejaculation: a qualitative assessment. Sex Med. 2023; 11(2):qfac020. Crossref, Medline, Google Scholar 8. . Social media and apps in urology. Curr Surg Rep. 2023; 10.1007/s40137-023-00366-9. Crossref, Medline, Google Scholar 9. . Comparing physician and artificial intelligence chatbot responses to patient questions posted to a public social media forum. JAMA Intern Med. 2023; 183(6):589-596. Crossref, Medline, Google Scholar 10. . Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models. PLOS Digital Health. 2023; 2(2):e0000198. Crossref, Medline, Google Scholar 11. Bridging the gap between urological research and patient understanding: the role of large language models in automated generation of layperson's summaries. Urol Pract. 2023; 10(5):436-443. Google Scholar 12. . New artificial intelligence ChatGPT performs poorly on the 2022 self-assessment study program for urology. Urol Pract. 2023; 10(4):409-415. Link, Google Scholar 13. . Quality of ChatGPT responses to questions related to liver transplantation. J Gastrointest Surg. 2023; 27(8):1716-1719. Crossref, Medline, Google Scholar 14. . Caution! AI bot has entered the patient chat: ChatGPT has limitations in providing accurate urologic healthcare advice. Urology. 2023; 180:278-284. Crossref, Medline, Google Scholar 15. . Readability assessment of online ophthalmic patient information. JAMA Ophthalmol. 2013; 131(12):1610-1616. Crossref, Medline, Google Scholar 16. . Readability of patient education materials on the American Association for Surgery of Trauma website. Arch Trauma Res. 2014; 3(2):e18161. Medline, Google Scholar 17. . Readability of online health information: a meta-narrative systematic review. Am J Med Qual. 2018; 33(5):487-492. Crossref, Medline, Google Scholar 18. . ChatGPT's ability to assess quality and readability of online medical information: evidence from a cross-sectional study. Cureus. 2023; 15(7):e42214. Medline, Google Scholar 19. . The Urology Care Foundation: addressing medical misinformation in urology. J Urol. 2022; 208(4):765-766. Link, Google Scholar 20. . Evaluating artificial intelligence responses to public health questions. JAMA Netw Open. 2023; 6(6):e2317517. Crossref, Medline, Google Scholar 21. Evaluating the effectiveness of artificial intelligence-powered large language models application in disseminating appropriate and readable health information in urology. J Urol. 2023; 210(4):688-694. Google Scholar 22. . A paradigm shift: online artificial intelligence platforms as an informational resource in bariatric surgery. Obes Surg. 2023; 33(8):2611-2614. Crossref, Medline, Google Scholar 23. . The utility of ChatGPT in generating patient-facing and clinical responses for melanoma. J Am Acad Dermatol. 2023; 89(3):602-604. Crossref, Medline, Google Scholar 24. . Artificial intelligence-based ChatGPT chatbot responses for patient and parent questions on vernal keratoconjunctivitis. Graefes Arch Clin Exp Ophthalmol. 2023; 261(10):3041-3043. Crossref, Medline, Google Scholar 25. . Literacy and health outcomes: a systematic review of the literature. J Gen Intern Med. 2004; 19(12):1228-1239. Crossref, Medline, Google Scholar 26. . An evaluation of the readability, quality, and accuracy of online health information regarding the treatment of hypospadias. J Pediatr Urol. 2019; 15(1):40.e1-40.e6. Crossref, Medline, Google Scholar 27. . Assessing the readability of online information about Achilles tendon ruptures. Foot Ankle Spec. 2020; 13(6):470-477. Crossref, Medline, Google Scholar 28. . Is ChatGPT an evidence-based doctor?. Eur Urol. 2023; 84(3):355-356. Crossref, Medline, Google Scholar 29. . Can ChatGPT, an artificial intelligence language model, provide accurate and high-quality patient information on prostate cancer?.Urology. 2023; 180:35-58. Crossref, Medline, Google Scholar 30. . In layman's terms: the power and problem of science communication. Disaster Med Public Health Prep. 2022; 16(6):2245-2247. Crossref, Google Scholar 31. . A critical review of ChatGPT as a potential substitute for diabetes educators. Cureus. 2023; 15(5):e38380. Medline, Google Scholar 32. . Empathy in the age of science disinformation: implications for healthcare quality. BMJ Lead. 2023; 10.1136/leader-2022-000716. Crossref, Medline, Google Scholar 33. . Harnessing generative artificial intelligence to improve efficiency among urologists: welcome ChatGPT. J Urol. 2023; 209(5):827-829. Link, Google Scholar Support: None. Conflict of Interest Disclosures: The Authors have no conflicts of interest to disclose. Ethics Statement: In lieu of review board approval, the principles of the Helsinki Declaration were followed. Author Contributions: Conception and design: Y.B.S., A.R.H., A.G., C.D.L., M.S.S., S.D.C.; Data analysis and interpretation: Y.B.S., A.R.H., A.G., C.D.L., E.R., S.D.C.; Data acquisition: Y.B.S., A.G., A.R.H., C.D.L., M.S.S.; Critical revision of the manuscript for scientific and factual content: A.G., C.D.L., E.R., M.S.S., S.D.C.; Drafting the manuscript: Y.B.S., A.R.H., S.D.C.; Statistical analysis: Y.B.S., A.R.H., A.G., E.R., S.D.C.; Supervision: C.D.L., M.S.S., S.D.C., M.S.S., S.D.C.; Reviewing and editing: A.G., A.R.H., E.R., C.D.L., M.S.S., S.D.C. Data Availability: Data will be made available upon reasonable request. © 2023 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetailsRelated articlesUrology Practice14 Nov 2023Editorial CommentUrology Practice14 Nov 2023Editorial Comment Volume 11 Issue 1 January 2024 Page: 87-94 Supplementary Materials Editorial Comment Video: Dr. Nguyen Advertisement Copyright & Permissions© 2023 by American Urological Association Education and Research, Inc.KeywordsChatGPTartificial intelligencemen’s healthpatient educationhealth literacyMetrics Author Information Yash B. Shah Department of Urology, Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, Pennsylvania More articles by this author Anushka Ghosh Department of Urology, Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, Pennsylvania More articles by this author Aaron R. Hochberg Department of Urology, Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, Pennsylvania More articles by this author Eli Rapoport Department of Urology, NYU Langone, New York, New York More articles by this author Costas D. Lallas Department of Urology, Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, Pennsylvania More articles by this author Mihir S. Shah Department of Urology, Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, Pennsylvania More articles by this author Seth D. Cohen Department of Urology, NYU Langone, New York, New York *Corresponding Author: Seth D. Cohen, MD, MPH, Department of Urology, NYU Grossman School of Medicine Director, Sexual Dysfunction Program, 222 E 41st St, 11th Floor, New York, NY 10017 ( E-mail Address: [email protected] More articles by this author Expand All Support: None. Conflict of Interest Disclosures: The Authors have no conflicts of interest to disclose. Ethics Statement: In lieu of review board approval, the principles of the Helsinki Declaration were followed. Author Contributions: Conception and design: Y.B.S., A.R.H., A.G., C.D.L., M.S.S., S.D.C.; Data analysis and interpretation: Y.B.S., A.R.H., A.G., C.D.L., E.R., S.D.C.; Data acquisition: Y.B.S., A.G., A.R.H., C.D.L., M.S.S.; Critical revision of the manuscript for scientific and factual content: A.G., C.D.L., E.R., M.S.S., S.D.C.; Drafting the manuscript: Y.B.S., A.R.H., S.D.C.; Statistical analysis: Y.B.S., A.R.H., A.G., E.R., S.D.C.; Supervision: C.D.L., M.S.S., S.D.C., M.S.S., S.D.C.; Reviewing and editing: A.G., A.R.H., E.R., C.D.L., M.S.S., S.D.C. Data Availability: Data will be made available upon reasonable request. Advertisement Advertisement PDF downloadLoading ...