Evidence for Health Decision Making — Beyond Randomized, Controlled Trials

A core principle of good public health practice is to base all policy decisions on the highest-quality scientific data, openly and objectively derived. 1 Determining whether data meet these conditions is difficult; uncertainty can lead to inaction by clinicians and public health decision makers.3][4][5][6][7][8] In this article, I describe the use of RCTs and alternative (and sometimes superior) data sources from the vantage point of public health, illustrate key limitations of RCTs, and suggest ways to improve the use of multiple data sources for health decision making.In large, well-designed trials, randomization evenly distributes known and unknown factors among control and intervention groups, reducing the potential for confounding.Despite their strengths, RCTs have substantial limitations.Although they can have strong internal validity, RCTs sometimes lack external validity; generalizations of findings outside the study population may be invalid. 2,4,6CTs usually do not have sufficient study periods or population sizes to assess duration of treatment effect (e.g., waning immunity of vaccines) or to identify rare but serious adverse effects of treatment, which often become evident during postmarketing surveillance and long-term follow-up but could not be practically assessed in an RCT.The increasingly high costs and time constraints of RCTs can also lead to reliance on surrogate markers that may not correlate well with the outcome of interest.Selection of high-risk groups increases the likelihood of having adequate numbers of end points, but these groups may not be relevant to the broader target populations.These limitations and the fact that RCTs often take years to plan, implement, and analyze reduce the ability of RCTs to keep pace with clinical innovations; new products and standards of care are often developed before earlier models complete evaluation.These limitations also affect the use of RCTs for urgent health issues, such as infectious disease outbreaks, for which public health decisions must be made quickly on the basis of limited and often imperfect available data.RCTs are also limited in their ability to assess the individualized effect of treatment, as can result from differences in surgical techniques, and are generally impractical for rare diseases.Many other data sources can provide valid evidence for clinical and public health action.Observational studies, including assessments of results from the From Atlanta, GA.