An Appraisal of the Carlisle-Stouffer-Fisher Method for Assessing Study Data Integrity and Fraud

医学 离群值 统计 无效假设 随机对照试验 随机化 统计假设检验 计量经济学 数学 外科
作者
Edward J. Mascha,Thomas R. Vetter,Jean François Pittet
出处
期刊:Anesthesia & Analgesia [Ovid Technologies (Wolters Kluwer)]
卷期号:125 (4): 1381-1385 被引量:19
标识
DOI:10.1213/ane.0000000000002415
摘要

Data fabrication and scientific misconduct have been recently uncovered in the anesthesia literature, partly via the work of John Carlisle. In a recent article in Anaesthesia , Carlisle analyzed 5087 randomized clinical trials from anesthesia and general medicine journals from 2000 to 2015. He concluded that in about 6% of studies, data comparing randomized groups on baseline variables, before the given intervention, were either too similar or dissimilar compared to that expected by usual sampling variability under the null hypothesis. Carlisle used the Stouffer-Fisher method of combining P values in Table 1 (the conventional table reporting baseline patient characteristics) for each study, then calculated trial P values and assessed whether they followed a uniform distribution across studies. Extreme P values targeted studies as likely to contain data fabrication or errors. In this Statistical Grand Rounds article, we explain Carlisle’s methods, highlight perceived limitations of the proposed approach, and offer recommendations. Our main findings are (1) independence was assumed between variables in a study, which is often false and would lead to “false positive” findings; (2) an “unusual” result from a trial cannot easily be concluded to represent fraud; (3) utilized cutoff values for determining extreme P values were arbitrary; (4) trials were analyzed as if simple randomization was used, introducing bias; (5) not all P values can be accurately generated from summary statistics in a Table 1, sometimes giving incorrect conclusions; (6) small numbers of P values to assess outlier status within studies is not reliable; (7) utilized method to assess deviations from expected distributions may stack the deck; (8) P values across trials assumed to be independent; (9) P value variability not accounted for; and (10) more detailed methods needed to understand exactly what was done. It is not yet known to what extent these concerns affect the accuracy of Carlisle’s results. We recommend that Carlisle’s methods be improved before widespread use (applying them to every manuscript submitted for publication). Furthermore, lack of data integrity and fraud should ideally be assessed using multiple simultaneous statistical methods to yield more confident results. More sophisticated methods are needed for nonrandomized trials, randomized trial data reported beyond Table 1, and combating growing fraudster sophistication. We encourage all authors to more carefully scrutinize their own reporting. Finally, we believe that reporting of suspected data fraud and integrity issues should be done more discretely and directly by the involved journal to protect honest authors from the stigma of being associated with potential fraud.

科研通智能强力驱动
Strongly Powered by AbleSci AI
更新
大幅提高文件上传限制,最高150M (2024-4-1)

科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
小蘑菇应助等待洙采纳,获得10
2秒前
大麦迪完成签到,获得积分20
2秒前
2秒前
希望天下0贩的0应助12345采纳,获得10
3秒前
杨tong发布了新的文献求助10
4秒前
BCKT完成签到,获得积分10
5秒前
5秒前
smmmy完成签到,获得积分10
7秒前
drwalyssa发布了新的文献求助10
8秒前
小巧曲奇完成签到,获得积分10
8秒前
wsazah完成签到,获得积分10
11秒前
12秒前
杲杲完成签到 ,获得积分10
13秒前
13秒前
sjj完成签到,获得积分20
15秒前
闪闪的垣完成签到,获得积分10
16秒前
爱静静应助一颗树采纳,获得10
17秒前
LJ发布了新的文献求助10
17秒前
17秒前
不忘初心发布了新的文献求助10
18秒前
19秒前
21秒前
24秒前
13504544355完成签到 ,获得积分10
25秒前
lwh发布了新的文献求助10
25秒前
26秒前
不忘初心完成签到,获得积分10
27秒前
12345完成签到,获得积分10
27秒前
淡淡的若冰应助了又柳采纳,获得10
28秒前
28秒前
Ava应助识途采纳,获得10
29秒前
xs发布了新的文献求助10
29秒前
烙饼完成签到,获得积分10
30秒前
33秒前
爱静静应助一颗树采纳,获得10
34秒前
shy完成签到,获得积分10
34秒前
xx完成签到,获得积分10
35秒前
雪花落在丛林完成签到,获得积分10
36秒前
37秒前
一二一发布了新的文献求助10
38秒前
高分求助中
Evolution 10000
юрские динозавры восточного забайкалья 800
English Wealden Fossils 700
Mantiden: Faszinierende Lauerjäger Faszinierende Lauerjäger 600
Distribution Dependent Stochastic Differential Equations 500
A new species of Coccus (Homoptera: Coccoidea) from Malawi 500
A new species of Velataspis (Hemiptera Coccoidea Diaspididae) from tea in Assam 500
热门求助领域 (近24小时)
化学 医学 生物 材料科学 工程类 有机化学 生物化学 物理 内科学 纳米技术 计算机科学 化学工程 复合材料 基因 遗传学 催化作用 物理化学 免疫学 量子力学 细胞生物学
热门帖子
关注 科研通微信公众号,转发送积分 3157519
求助须知:如何正确求助?哪些是违规求助? 2808900
关于积分的说明 7878979
捐赠科研通 2467322
什么是DOI,文献DOI怎么找? 1313355
科研通“疑难数据库(出版商)”最低求助积分说明 630395
版权声明 601919