Real-world studies no substitute for RCTs in establishing efficacy

We live in the real world, so it is reasonable to expect that data collected from the real world should help identify effective therapies. Indeed, rapid increases in the availability of registries, electronic health records, and insurance claims, and the ability to access, process, link, and analyse data from these sources at fairly low cost lend support for calls to replace randomised controlled trials (RCTs) with so-called real-world studies to establish the efficacy of a therapy, 1 National Academies of Sciences Engineering, and MedicineReal-world evidence generation and evaluation of therapeutics: proceedings of a workshop. The National Academies Press, Washington, DC2017 Google Scholar , 2 Franklin JM Schneeweiss S When and how can real world data analyses substitute for randomized controlled trials?. Clin Pharmacol Ther. 2017; 102: 924-933 Crossref PubMed Scopus (153) Google Scholar particularly for common serious diseases with abundant, easily collected data such as diabetes. 3 Cavender MA Norhammar A Birkeland KI et al. SGLT-2 Inhibitors and cardiovascular risk: an analysis of CVD-REAL. J Am Coll Cardiol. 2018; 71: 2497-2506 Crossref PubMed Scopus (89) Google Scholar This push is driven partly by the need to show payers that therapies are working and are therefore of value when used in the real world. 4 Shaywitz D Will real world performance replace RCTs as healthcare's most important standard?. Forbes. May 11, 2018; https://www.forbes.com/sites/davidshaywitz/2018/05/11/will-real-world-performance-replace-rcts-as-healthcares-mostimportant-standard/#46a696ee3557Date accessed: October 25, 2018 Google Scholar Other driving factors include the industry's wish to reduce costs and time to get results, a mistaken belief that real-world data are somehow more relevant than RCT data for establishing efficacy, and the ease and speed with which registry data can be accessed and publications generated. However, even with the use of sophisticated methods to address various sources of bias, 5 Gill J Prasad V Improving observational studies in the era of big data. Lancet. 2018; 392: 716-717 Summary Full Text Full Text PDF PubMed Scopus (19) Google Scholar the absence of randomisation 6 Rush CJ Campbell RT Jhund PS Petrie MC McMurray JJV Association is not causation: treatment effects cannot be estimated from observational data in heart failure. Eur Heart J. 2018; 39: 3417-3438 Crossref PubMed Scopus (34) Google Scholar precludes protection from confounding and can lead payers and clinicians alike to erroneously infer that a therapy is beneficial or harmful. The importance of randomised vs non-randomised trialsWe thank Hertzel Gerstein and colleagues1 for reminding us of the importance of randomised controlled trials (RCTs). However, RCTs require uncertainty about the benefits of an intervention, and once an intervention has already become health policy, ethical issues with doing an RCT arise. Full-Text PDF The importance of randomised vs non-randomised trialsHertzel Gerstein and colleagues1 propose that estimates of treatment effects with a relative risk (RR) of more than 4 might show that randomised controlled trials (RCTs) are not needed because confounders are less likely to obscure the true treatment effects when effect sizes are this large. Other studies have suggested different thresholds of an RR of 10 or higher,2 or 5 or higher (or RR <0·2)3 to avoid an RCT. We agree that larger effects are less likely to be explained by confounding factors, but regulatory agencies appear to have no explicit or implicit thresholds for approval without requiring further testing in RCTs. Full-Text PDF The importance of randomised vs non-randomised trials – Authors' replyWe thank the correspondents for their responses to our Comment.1 Full-Text PDF