清晨好,您是今天最早来到科研通的研友!由于当前在线用户较少,发布求助请尽量完整地填写文献信息,科研通机器人24小时在线,伴您科研之路漫漫前行!

Retrieval-Augmented Generation–Enabled GPT-4 for Clinical Trial Screening

临床试验 计算机科学 计算生物学 医学 情报检索 环境科学 生物 内科学
作者
Ozan Ünlü,Jiyeon Shin,Charlotte Mailly,Michael Oates,Michela Tucci,Matthew Varugheese,Kavishwar B. Wagholikar,Fei Wang,Benjamin M. Scirica,Alexander Blood,Samuel Aronson
出处
期刊: [Massachusetts Medical Society]
卷期号:1 (7) 被引量:9
标识
DOI:10.1056/aioa2400181
摘要

BackgroundScreening participants in clinical trials is an error-prone and labor-intensive process that requires significant time and resources. Large language models such as generative pretrained transformer 4 (GPT-4) present an opportunity to enhance the screening process with advanced natural language processing. This study evaluates the utility of a Retrieval-Augmented Generation (RAG)–enabled GPT-4 system to improve the accuracy, efficiency, and reliability of screening for a trial involving patients with symptomatic heart failure.MethodsThe ongoing Co-Operative Program for Implementation of Optimal Therapy in Heart Failure (COPILOT-HF; ClinicalTrials.gov number, NCT05734690) trial identifies potential participants through electronic health record (EHR) queries followed by manual reviews by trained but nonlicensed study staff. To determine patient eligibility for the COPILOT-HF study that is not identifiable by structured EHR queries, we developed RAG-Enabled Clinical Trial Infrastructure for Inclusion Exclusion Review (RECTIFIER), a clinical note–based, question-answering system powered by RAG and GPT-4. We used clinical notes on 100, 282, and 1894 patients for development, validation, and test datasets, respectively. An expert clinician conducted a blinded review to establish "gold standard" answers to 13 target criteria questions. We calculated performance metrics (sensitivity, specificity, accuracy, and Matthews correlation coefficient [MCC]) in determining patient eligibility for each target criterion and for each of four screening methods (study staff, RECTIFIER with a single-question strategy, RECTIFIER with a combined-question strategy, and RECTIFIER with GPT-3.5 instead of GPT-4).ResultsThe RECTIFIER and COPILOT-HF study staff's answers closely aligned with the expert clinicians' answers across the target criteria, with accuracy ranging between 97.9% and 100% (MCC, 0.837 and 1) for RECTIFIER and between 91.7% and 100% (MCC, 0.644 and 1) for the study staff. RECTIFIER performed better than the study staff in determining symptomatic heart failure, with an accuracy of 97.9% versus 91.7% and an MCC of 0.924 versus 0.721, respectively. Overall, the sensitivity and specificity for determining patient eligibility with RECTIFIER were 92.3% and 93.9%, respectively, and 90.1% and 83.6% with the study staff. With RECTIFIER, the single-question approach to determining eligibility resulted in an average cost of 11 cents per patient, and the combined-question approach resulted in an average cost of 2 cents per patient.ConclusionsLarge language model–based solutions such as RECTIFIER can significantly enhance clinical trial screening performance and reduce costs by automating the screening process. However, integrating such technologies requires careful consideration of potential hazards and should include safeguards such as final clinician review. (Funded by the Accelerator for Clinical Transformation [ACT]; ClinicalTrials.gov number, NCT05734690.)

科研通智能强力驱动
Strongly Powered by AbleSci AI
科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
Maestro_S的应助被dianeil采纳,获得10
4秒前
Maestro_S的应助被dianeil采纳,获得10
18秒前
开心的灭龙完成签到,获得积分10
27秒前
sissiarno完成签到,获得积分0
40秒前
灵巧孤菱完成签到,获得积分10
42秒前
45秒前
直率的大门完成签到,获得积分10
50秒前
高大凌旋完成签到,获得积分10
55秒前
chenpy1990完成签到 ,获得积分10
1分钟前
1分钟前
dianeil发布了新的文献求助10
1分钟前
可爱的函函的应助被stanfordlee采纳,获得10
1分钟前
迷你的金鱼完成签到,获得积分10
1分钟前
1分钟前
现代的初之完成签到,获得积分10
1分钟前
CC完成签到,获得积分20
1分钟前
典雅依玉完成签到,获得积分10
1分钟前
老戎完成签到 ,获得积分10
1分钟前
arsenal完成签到 ,获得积分10
1分钟前
lay完成签到,获得积分10
1分钟前
完美的睿渊完成签到,获得积分10
1分钟前
aimynora完成签到 ,获得积分10
2分钟前
潇洒醉冬完成签到,获得积分10
2分钟前
瘦瘦的涫完成签到,获得积分10
2分钟前
Lemenchichi完成签到,获得积分10
2分钟前
MassTestA1的应助被MassTestA1采纳,获得50
2分钟前
qin202569完成签到,获得积分20
2分钟前
宝石发布了新的文献求助10
2分钟前
追寻的凌香完成签到,获得积分10
2分钟前
2分钟前
2分钟前
stanfordlee发布了新的文献求助10
2分钟前
vbnn完成签到 ,获得积分10
2分钟前
热情的梦秋完成签到,获得积分10
3分钟前
LINDENG2004完成签到 ,获得积分10
3分钟前
隐形的灵薇完成签到,获得积分10
3分钟前
仁爱采蓝完成签到,获得积分10
3分钟前
xfy发布了新的文献求助200
3分钟前
Keyh完成签到,获得积分10
3分钟前
4分钟前
高分求助中
(应助此贴封号)【重要!!请各用户(尤其是新用户)详细阅读】【科研通的精品贴汇总】 10000
拉伸高分子熔体中缠结行为的模拟和小角中子散射研究 1700
The World Atlas of Wine 9th edition 1000
奈特基础组织学:配套组织病理学(第4版)(Netter's Essential Histology With Correlated Histopathology) 800
八旗制度の研究 600
高分子基功能界面水蒸发材料与应用研究 500
外场增强的水汽发电研究 500
热门求助领域 (近24小时)
化学 材料科学 医学 生物 纳米技术 计算机科学 工程类 有机化学 化学工程 生物化学 物理 复合材料 内科学 催化作用 光电子学 人工智能 细胞生物学 无机化学 物理化学 数学
热门帖子
关注 科研通微信公众号,转发送积分 8185453
求助须知:如何正确求助?哪些是违规求助? 9574168
关于积分的说明 21834449
捐赠科研通 7782987
什么是DOI,文献DOI怎么找? 3434570
关于科研通互助平台的介绍 2558881
邀请新用户注册赠送积分活动 2481299