Pilot Trial Characteristics, Postpilot Design Modifications, and Feasibility of Full-Scale Trials

Importance Pilot trials often lead to study design changes in subsequent full-scale trials. Yet, it remains unclear whether these modifications improve the feasibility of the larger trial. Objective To compare feasibility estimates between pilot and full-scale trials and identify pilot trial characteristics and modifications associated with equivalent or improved feasibility in the full-scale trial. Design, Setting, and Participants This cohort study used pilot trials published between January 2005 and December 2018 and their corresponding full-scale trials. PubMed was searched for trials on February 19, 2022. Exposures Pilot trial characteristics and postpilot trial design modifications. Main Outcomes and Measures The outcome of interest was difference in 3 feasibility parameters: successful screening probability, enrollment rate, and retention probability. These metrics were defined as equivalent or improved if the full-scale trial’s estimate was within or exceeding 10% of the pilot trial’s estimate. Results A total of 249 pairs of pilot and full-scale trials were analyzed, with 43%, 77%, and 82% of full-scale trials having equivalent or improved successful screening probabilities, enrollment rates, and retention probabilities, respectively. When pilot trials used feasibility progression criteria (relative risk [RR], 1.94; 95% CI, 1.02-5.97) and maintained masking for participants (RR, 1.82; 95% CI, 1.04-4.33) or health care practitioners (RR, 1.81; 95% CI, 1.03-3.97) consistent with the full-scale trial, the likelihood of achieving equivalent or improved screening success in full-scale trials increased. Increasing study sites after the pilot was associated with higher likelihood of equivalent or improved enrollment rates (RR, 1.03; 95% CI, 1.01-1.08). Adding extra content to the intervention (RR, 0.82; 95% CI, 0.66-0.98), changing to active control (RR, 0.74; 95% CI, 0.48-0.99), administrating the control treatment more frequently (RR, 0.60; 95% CI, 0.29-0.93), different follow-up lengths (RR, 0.85; 95% CI, 0.73-0.97), and more follow-up visits (RR, 0.86; 95% CI, 0.75-0.98) were associated with lower likelihood of equivalent or improved retention probability in the full-scale trial. Conclusions and relevance In this cohort study of pilot and full-scale trial pairs, pilot trial characteristics and postpilot modifications had varying associations with full-scale trial’s feasibility. If full-scale trials planned for masking, it was desirable to use it in the pilot. Modifications increasing participant burden might decrease full-scale trial feasibility. Trialists and funders should consider both pilot trial data and proposed design changes when assessing full-scale trials.