Evidence-based potential of generative artificial intelligence large language models in orthodontics: a comparative study of ChatGPT, Google Bard, and Microsoft Bing

印为红字的 Microsoft excel 计算机科学 清晰 人工智能 统计 机器学习 自然语言处理 心理学 数学 数学教育 生物化学 化学 操作系统
作者
Miltiadis A. Makrygiannakis,Kostis Giannakopoulos,Eleftherios G. Kaklamanos
出处
期刊:European Journal of Orthodontics [Oxford University Press]
标识
DOI:10.1093/ejo/cjae017
摘要

The increasing utilization of large language models (LLMs) in Generative Artificial Intelligence across various medical and dental fields, and specifically orthodontics, raises questions about their accuracy.This study aimed to assess and compare the answers offered by four LLMs: Google's Bard, OpenAI's ChatGPT-3.5, and ChatGPT-4, and Microsoft's Bing, in response to clinically relevant questions within the field of orthodontics.Ten open-type clinical orthodontics-related questions were posed to the LLMs. The responses provided by the LLMs were assessed on a scale ranging from 0 (minimum) to 10 (maximum) points, benchmarked against robust scientific evidence, including consensus statements and systematic reviews, using a predefined rubric. After a 4-week interval from the initial evaluation, the answers were reevaluated to gauge intra-evaluator reliability. Statistical comparisons were conducted on the scores using Friedman's and Wilcoxon's tests to identify the model providing the answers with the most comprehensiveness, scientific accuracy, clarity, and relevance.Overall, no statistically significant differences between the scores given by the two evaluators, on both scoring occasions, were detected, so an average score for every LLM was computed. The LLM answers scoring the highest, were those of Microsoft Bing Chat (average score = 7.1), followed by ChatGPT 4 (average score = 4.7), Google Bard (average score = 4.6), and finally ChatGPT 3.5 (average score 3.8). While Microsoft Bing Chat statistically outperformed ChatGPT-3.5 (P-value = 0.017) and Google Bard (P-value = 0.029), as well, and Chat GPT-4 outperformed Chat GPT-3.5 (P-value = 0.011), all models occasionally produced answers with a lack of comprehensiveness, scientific accuracy, clarity, and relevance.The questions asked were indicative and did not cover the entire field of orthodontics.Language models (LLMs) show great potential in supporting evidence-based orthodontics. However, their current limitations pose a potential risk of making incorrect healthcare decisions if utilized without careful consideration. Consequently, these tools cannot serve as a substitute for the orthodontist's essential critical thinking and comprehensive subject knowledge. For effective integration into practice, further research, clinical validation, and enhancements to the models are essential. Clinicians must be mindful of the limitations of LLMs, as their imprudent utilization could have adverse effects on patient care.
最长约 10秒,即可获得该文献文件

科研通智能强力驱动
Strongly Powered by AbleSci AI
科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
1秒前
1秒前
AireenBeryl531举报zeng的求助涉嫌违规
1秒前
yanyan完成签到,获得积分10
1秒前
2秒前
奥美拉完成签到,获得积分10
2秒前
呜呜呜呜呜完成签到,获得积分10
3秒前
4秒前
5秒前
在搬砖的大美完成签到,获得积分10
5秒前
幻涯完成签到,获得积分20
6秒前
NexusExplorer的应助被飞快的孱采纳,获得10
6秒前
6秒前
6秒前
6秒前
6秒前
6秒前
6秒前
6秒前
6秒前
6秒前
6秒前
6秒前
7秒前
7秒前
lin完成签到,获得积分10
8秒前
奥美拉发布了新的文献求助10
8秒前
无私的蛋挞完成签到 ,获得积分10
9秒前
乐观的醉香完成签到,获得积分10
9秒前
cc2004bj发布了新的文献求助10
9秒前
在水一方的应助被ccc采纳,获得10
11秒前
英勇的人生完成签到,获得积分10
11秒前
CAOHOU的应助被实验室采纳,获得10
11秒前
Hello的应助被hd采纳,获得30
11秒前
幻涯发布了新的文献求助10
11秒前
天才美少女骚猪完成签到,获得积分10
12秒前
mao发布了新的文献求助10
13秒前
13秒前
13秒前
13秒前
高分求助中
(应助此贴封号)【重要!!请各用户(尤其是新用户)详细阅读】【科研通的精品贴汇总】 10000
拉伸高分子熔体中缠结行为的模拟和小角中子散射研究 1700
The World Atlas of Wine 9th edition 1000
奈特基础组织学:配套组织病理学(第4版)(Netter's Essential Histology With Correlated Histopathology) 800
八旗制度の研究 600
Boundaries and Bodhisattvas 500
高分子基功能界面水蒸发材料与应用研究 500
热门求助领域 (近24小时)
化学 材料科学 医学 生物 纳米技术 计算机科学 工程类 有机化学 化学工程 生物化学 物理 复合材料 内科学 催化作用 光电子学 人工智能 细胞生物学 无机化学 物理化学 数学
热门帖子
关注 科研通微信公众号,转发送积分 8181949
求助须知:如何正确求助?哪些是违规求助? 9571702
关于积分的说明 21819272
捐赠科研通 7778399
什么是DOI,文献DOI怎么找? 3433572
关于科研通互助平台的介绍 2557769
邀请新用户注册赠送积分活动 2479630