Supervised machine learning compared to large language models for identifying functional seizures from medical records

置信区间 接收机工作特性 逻辑回归 脑电图 卡帕 神经影像学 癫痫 医学 惊厥 机器学习 心理学 内科学 听力学 人工智能 精神科 计算机科学 数学 几何学
作者
Wesley T. Kerr,Katherine N. McFarlane,Gabriela Figueiredo Pucci,Danielle R. Carns,Alex Israel,Lianne Vighetti,Page B. Pennell,John M. Stern,Zongqi Xia,Yanshan Wang
出处
期刊:Epilepsia [Wiley]
标识
DOI:10.1111/epi.18272
摘要

Abstract Objective The Functional Seizures Likelihood Score (FSLS) is a supervised machine learning–based diagnostic score that was developed to differentiate functional seizures (FS) from epileptic seizures (ES). In contrast to this targeted approach, large language models (LLMs) can identify patterns in data for which they were not specifically trained. To evaluate the relative benefits of each approach, we compared the diagnostic performance of the FSLS to two LLMs: ChatGPT and GPT‐4. Methods In total, 114 anonymized cases were constructed based on patients with documented FS, ES, mixed ES and FS, or physiologic seizure‐like events (PSLEs). Text‐based data were presented in three sequential prompts to the LLMs, showing the history of present illness (HPI), electroencephalography (EEG) results, and neuroimaging results. We compared the accuracy (number of correct predictions/number of cases) and area under the receiver‐operating characteristic (ROC) curves (AUCs) of the LLMs to the FSLS using mixed‐effects logistic regression. Results The accuracy of FSLS was 74% (95% confidence interval [CI] 65%–82%) and the AUC was 85% (95% CI 77%–92%). GPT‐4 was superior to both the FSLS and ChatGPT ( p <.001), with an accuracy of 85% (95% CI 77%–91%) and AUC of 87% (95% CI 79%–95%). Cohen's kappa between the FSLS and GPT‐4 was 40% (fair). The LLMs provided different predictions on different days when the same note was provided for 33% of patients, and the LLM's self‐rated certainty was moderately correlated with this observed variability (Spearman's rho 2 : 30% [fair, ChatGPT] and 63% [substantial, GPT‐4]). Significance Both GPT‐4 and the FSLS identified a substantial subset of patients with FS based on clinical history. The fair agreement in predictions highlights that the LLMs identified patients differently from the structured score. The inconsistency of the LLMs' predictions across days and incomplete insight into their own consistency was concerning. This comparison highlights both benefits and cautions about how machine learning and artificial intelligence could identify patients with FS in clinical practice.
最长约 10秒,即可获得该文献文件

科研通智能强力驱动
Strongly Powered by AbleSci AI
更新
PDF的下载单位、IP信息已删除 (2025-6-4)

科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
FashionBoy应助风趣的灵枫采纳,获得10
刚刚
1秒前
yihuiqing发布了新的文献求助10
2秒前
likke发布了新的文献求助10
2秒前
嗝嗝发布了新的文献求助10
2秒前
小兔狸花昕完成签到,获得积分20
3秒前
IRer79完成签到,获得积分10
5秒前
彭于晏应助noah采纳,获得10
5秒前
青年才俊发布了新的文献求助10
5秒前
zhu发布了新的文献求助10
6秒前
波波蛋完成签到,获得积分10
6秒前
7秒前
8秒前
9秒前
maguodrgon发布了新的文献求助10
12秒前
科研通AI5应助shell采纳,获得80
12秒前
波波蛋发布了新的文献求助10
12秒前
13秒前
九鹤完成签到,获得积分10
13秒前
jsy完成签到,获得积分10
13秒前
量子星尘发布了新的文献求助10
14秒前
旷野发布了新的文献求助10
14秒前
14秒前
Owen应助浮云521采纳,获得10
14秒前
浮游应助muming采纳,获得10
14秒前
酷炫的凤妖完成签到 ,获得积分10
15秒前
16秒前
16秒前
18秒前
18秒前
18秒前
19秒前
20秒前
浮游应助勤劳的莆采纳,获得10
20秒前
21秒前
霜降发布了新的文献求助10
22秒前
jsy发布了新的文献求助10
22秒前
daring完成签到,获得积分10
22秒前
22秒前
上官若男应助maguodrgon采纳,获得10
24秒前
高分求助中
(应助此贴封号)【重要!!请各用户(尤其是新用户)详细阅读】【科研通的精品贴汇总】 10000
Zeolites: From Fundamentals to Emerging Applications 1500
Architectural Corrosion and Critical Infrastructure 1000
Early Devonian echinoderms from Victoria (Rhombifera, Blastoidea and Ophiocistioidea) 1000
By R. Scott Kretchmar - Practical Philosophy of Sport and Physical Activity - 2nd (second) Edition: 2nd (second) Edition 666
Energy-Size Reduction Relationships In Comminution 500
Principles Of Comminution, I-Size Distribution And Surface Calculations 500
热门求助领域 (近24小时)
化学 医学 生物 材料科学 工程类 有机化学 内科学 生物化学 物理 计算机科学 纳米技术 遗传学 基因 复合材料 化学工程 物理化学 病理 催化作用 免疫学 量子力学
热门帖子
关注 科研通微信公众号,转发送积分 4941339
求助须知:如何正确求助?哪些是违规求助? 4207390
关于积分的说明 13077624
捐赠科研通 3986257
什么是DOI,文献DOI怎么找? 2182529
邀请新用户注册赠送积分活动 1198125
关于科研通互助平台的介绍 1110387