Network Meta-analysis of Ulcerative Colitis Pharmacotherapies: Carryover Effects From Induction and Bias of the Results

In their network meta-analyses (NMAs) of treatments for ulcerative colitis (UC), Singh et al1Singh S. et al.Clin Gastroenterol Hepatol. 2020; 18: 2179-2191Abstract Full Text Full Text PDF PubMed Scopus (54) Google Scholar did not take into account a complication associated with studies that rerandomized patients for the maintenance phase: differential carryover effects from induction can bias the results. In those studies, patients who responded to induction were rerandomized to maintenance treatments that included placebo. If, however, carryover effects from induction differ substantially among active treatments, the effects of those treatments, relative to placebo, are not comparable. Placebo rates in the rerandomized maintenance studies suggest that ustekinumab provided greater carryover effects from induction than the other treatments. In the biologic-nonfailure populations, 31.0% in the ustekinumab study,2Sands B.E. et al.N Engl J Med. 2019; 381: 1201-1214Crossref PubMed Scopus (240) Google Scholar 20.5% in the vedolizumab study,3Feagan B.G. et al.Clin Gastroenterol Hepatol. 2017; 15: 229-239Abstract Full Text Full Text PDF PubMed Scopus (92) Google Scholar 15.6% in the golimumab study,4Gibson P.R. et al.Clin Transl Gastroenterol. 2016; 7: e168Crossref PubMed Google Scholar and 11.0% in the tofacitinib study5Dubinsky MC, et al. World Congress of Gastroenterology at ACG2017 Meeting 2017; Program No. P449.Google Scholar were in clinical remission after responding to induction and receiving placebo maintenance (chi-square test; P < .001). In the biologic-failure populations, 17.0% in the ustekinumab study, 11.2% in the tofacitinib study, and 5.3% in the vedolizumab study were in clinical remission (P < .001, not evaluated in the golimumab study). This trend was previously observed in randomized-withdrawal studies in patients with Crohn's disease,6Varu A. et al.Curr Med Res Opin. 2019; 35: 733-756Crossref PubMed Scopus (4) Google Scholar suggesting that it is a characteristic of ustekinumab induction treatment and not related to a unique aspect of the UC study population. Because of these differences in carryover effects, the "placebo" groups in the rerandomized maintenance studies cannot serve as a common comparator. They are not true placebo groups, because they consist of patients who responded to different induction treatments and were evaluated after receiving placebo maintenance. Thus, the NMA of rerandomized maintenance studies underestimated the incremental benefit that ustekinumab maintenance provided over placebo and therefore relative to other treatments. The difference between ustekinumab and placebo maintenance did not reach statistical significance in the NMA (odds ratio [95% confidence interval] 2.46 [0.61–9.88] for clinical remission and 2.62 [0.95–7.23] for endoscopic improvement) (Supplementary Table 2B). These estimates directly contradict the actual results from the UNIFI study, in which differences between ustekinumab and placebo maintenance were statistically significant and clinically meaningful.2Sands B.E. et al.N Engl J Med. 2019; 381: 1201-1214Crossref PubMed Scopus (240) Google Scholar The failure to corroborate the original results from this large, prospective, randomized, pivotal study underscores the flawed methodology of this NMA. Overall, the NMA of rerandomized maintenance studies was based on a thin network of 6 trials and 4 UC treatments versus placebo, and the results indicate that the network contained substantial heterogeneity. For each direct comparison of clinical remission in the pairwise meta-analysis (MA) (Supplemental Figure 5A), the corresponding odds ratio in the NMA has a much wider confidence interval (Supplementary Table 2B). One would expect the NMA not to have such wide confidence intervals, because it can draw from the whole network to estimate the heterogeneity variance. However, the comparison between the pairwise MA and the NMA is difficult to interpret because the pairwise MA used the Mantel-Haenszel fixed-effect method, and the assumptions used in the software to estimate the heterogeneity variance in the NMA were not reported. Comparisons with results from the DerSimonian-Laird random-effects method would have been more informative, along with a detailed description of the NMA methods and publication of the code used in the analysis. Heterogeneity in randomized-withdrawal trial designs can be accounted for by recalculating data to mimic a treat-through design, maintaining the initial randomized treatment groups from induction, accounting for initial and delayed responders to induction treatment, and using true placebo as a common comparator. Welty et al7Welty M. et al.Curr Med Res Opin. 2020; 36: 595-606Crossref PubMed Scopus (7) Google Scholar used this approach in an NMA of UC studies and found that ustekinumab had greater probabilities of clinical response, clinical remission, and endoscopic improvement through 1 year in patients who had not failed biologics than adalimumab, vedolizumab, golimumab, and infliximab. This approach allowed use of a larger network that consisted of studies with treat-through designs and studies with randomized-withdrawal designs. Notably, even when VARSITY data8Sands B.E. et al.N Engl J Med. 2019; 381: 1215-1226Crossref PubMed Scopus (170) Google Scholar were excluded from that NMA, the comparison of vedolizumab versus adalimumab showed a similar direction to VARSITY, further supporting the validity of the NMA methodology. The underlying assumptions in the NMAs of Singh et al1Singh S. et al.Clin Gastroenterol Hepatol. 2020; 18: 2179-2191Abstract Full Text Full Text PDF PubMed Scopus (54) Google Scholar have important implications for treatment of patients with UC. The results were used to formulate the recently published practice guidelines from the American Gastroenterology Association.9Feuerstein J.D. et al.Gastroenterology. 2020; 158: 1450-1461Abstract Full Text Full Text PDF PubMed Scopus (60) Google Scholar Payors may use them to inform formulary decisions that limit patients' access. NMAs are valuable in the absence of direct comparisons of treatments in head-to-head trials, but assumptions implicit in their design must be rigorously scrutinized. ReplyClinical Gastroenterology and HepatologyVol. 19Issue 10PreviewWe appreciate Naennes et al's interest in our study. The primary emphasis and inferences from our network meta-analyses,1 and in the American Gastroenterological Association guidelines on the management of moderate to severe ulcerative colitis, were drawn from trials of induction therapy.2,3 We acknowledge limitations in interpreting findings from trials of maintenance therapy because of differences in trial design. Full-Text PDF