Age against the machine—susceptibility of large language models to cognitive impairment: cross sectional analysis

蒙特利尔认知评估 斯特罗普效应 认知 考试(生物学) 心理学 认知心理学 执行职能 认知障碍 精神科 古生物学 生物
作者
Roy Dayan,Benjamin Uliel,Gal Koplewitz
标识
DOI:10.1136/bmj-2024-081948
摘要

Abstract Objective To evaluate the cognitive abilities of the leading large language models and identify their susceptibility to cognitive impairment, using the Montreal Cognitive Assessment (MoCA) and additional tests. Design Cross sectional analysis. Setting Online interaction with large language models via text based prompts. Participants Publicly available large language models, or “chatbots”: ChatGPT versions 4 and 4o (developed by OpenAI), Claude 3.5 “Sonnet” (developed by Anthropic), and Gemini versions 1 and 1.5 (developed by Alphabet). Assessments The MoCA test (version 8.1) was administered to the leading large language models with instructions identical to those given to human patients. Scoring followed official guidelines and was evaluated by a practising neurologist. Additional assessments included the Navon figure, cookie theft picture, Poppelreuter figure, and Stroop test. Main outcome measures MoCA scores, performance in visuospatial/executive tasks, and Stroop test results. Results ChatGPT 4o achieved the highest score on the MoCA test (26/30), followed by ChatGPT 4 and Claude (25/30), with Gemini 1.0 scoring lowest (16/30). All large language models showed poor performance in visuospatial/executive tasks. Gemini models failed at the delayed recall task. Only ChatGPT 4o succeeded in the incongruent stage of the Stroop test. Conclusions With the exception of ChatGPT 4o, almost all large language models subjected to the MoCA test showed signs of mild cognitive impairment. Moreover, as in humans, age is a key determinant of cognitive decline: “older” chatbots, like older patients, tend to perform worse on the MoCA test. These findings challenge the assumption that artificial intelligence will soon replace human doctors, as the cognitive impairment evident in leading chatbots may affect their reliability in medical diagnostics and undermine patients’ confidence.

科研通智能强力驱动
Strongly Powered by AbleSci AI
更新
大幅提高文件上传限制,最高150M (2024-4-1)

科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
8R60d8应助安静的元枫采纳,获得10
刚刚
不散,不见完成签到,获得积分10
1秒前
黄小慧发布了新的文献求助10
2秒前
顺利的白昼完成签到,获得积分10
2秒前
Lucas应助长隆采纳,获得10
2秒前
一杯CC发布了新的文献求助10
2秒前
3秒前
3秒前
4秒前
微笑铅笔应助难摧采纳,获得10
4秒前
5秒前
7秒前
研友_VZG7GZ应助郭果儿采纳,获得10
7秒前
ABS驳回了脑洞疼应助
7秒前
7秒前
7秒前
微笑世开完成签到 ,获得积分10
8秒前
小马甲应助冷傲的芾采纳,获得10
8秒前
song发布了新的文献求助10
8秒前
KK发布了新的文献求助10
9秒前
LiuXiaoJie发布了新的文献求助10
9秒前
善学以致用应助mygod采纳,获得10
9秒前
哈哈发布了新的文献求助20
10秒前
背书强发布了新的文献求助20
11秒前
123lx发布了新的文献求助10
11秒前
yuanling完成签到 ,获得积分10
12秒前
斯文败类应助细腻盼烟采纳,获得10
12秒前
12秒前
CipherSage应助王千鹤采纳,获得10
13秒前
一杯CC完成签到,获得积分10
14秒前
15秒前
星辰大海应助jiayou采纳,获得10
15秒前
song完成签到,获得积分10
16秒前
kekeke完成签到,获得积分10
18秒前
科目三应助ranan采纳,获得10
18秒前
18秒前
19秒前
20秒前
zho应助······采纳,获得10
20秒前
张宝完成签到,获得积分10
21秒前
高分求助中
The late Devonian Standard Conodont Zonation 2000
Nickel superalloy market size, share, growth, trends, and forecast 2023-2030 2000
The Lali Section: An Excellent Reference Section for Upper - Devonian in South China 1500
Smart but Scattered: The Revolutionary Executive Skills Approach to Helping Kids Reach Their Potential (第二版) 1000
Very-high-order BVD Schemes Using β-variable THINC Method 830
Mantiden: Faszinierende Lauerjäger Faszinierende Lauerjäger 800
PraxisRatgeber: Mantiden: Faszinierende Lauerjäger 800
热门求助领域 (近24小时)
化学 医学 生物 材料科学 工程类 有机化学 生物化学 物理 内科学 纳米技术 计算机科学 化学工程 复合材料 基因 遗传学 催化作用 物理化学 免疫学 量子力学 细胞生物学
热门帖子
关注 科研通微信公众号,转发送积分 3247867
求助须知:如何正确求助?哪些是违规求助? 2891062
关于积分的说明 8266031
捐赠科研通 2559319
什么是DOI,文献DOI怎么找? 1388095
科研通“疑难数据库(出版商)”最低求助积分说明 650694
邀请新用户注册赠送积分活动 627581