Network meta‐analysis: methodological points for readers, authors and reviewers

Network meta-analysis (NMA) is a useful statistical method that allows comparison of multiple treatments to be considered in a single analysis by combining direct with indirect evidence. The BJD has seen an increase in submissions of systematic reviews employing NMA over the past couple of years;1 therefore, we now provide methodological guidance to help authors submit a high-quality NMA. Direct evidence is often obtained from randomized controlled trials while indirect evidence can be mathematically deduced when two or more interventions have been compared with a common comparator. For example, in a recent Cochrane Database systematic review and NMA, 20 systemic treatments for moderate-to-severe psoriasis were considered.2 The relative effect of infliximab vs. secukinumab – for which no study is available – was estimated indirectly via comparisons with placebo (Figure 1). NMA also allows one to rank treatments, thus answering an important question for physicians, patients and guideline authors: among all available treatments, which works best? Given the growing spike in publications related to NMA, concerns have emerged regarding their methodological quality.3 To ensure validity of findings, it is fundamental that authors accurately plan, conduct and report a NMA. This includes the formulation of a precise, clinically pertinent research question, the conduct of a thorough systematic review, assessment of the assumptions of NMA, transparency and comprehensive presentation of results, and the evaluation of risk of bias and certainty of the evidence. A protocol, which outlines these stages, needs to be prospectively registered. Authors should follow the PRISMA extension statement for systematic reviews with NMA to ensure comprehensiveness and transparency of reporting.4 A well-formulated question is crucial in guiding authors throughout the NMA, from the definition of eligibility criteria to the reporting of findings, and will help to determine which populations and treatments to include in the network, and thus the shape the network of evidence may take. Decisions of whether the different interventions should be evaluated as individual drugs, specific doses, or lumped into drug classes need to be made in consideration of the research question and the underlying assumptions, notably the assumption of transitivity.5 Transitivity refers to the validity of carrying out indirect comparisons via an intermediate treatment and is a fundamental assumption of NMAs. It assumes there are no major differences between the included studies regarding all important factors that may affect the outcome, such as patient characteristics and disease severity. For example, trials involving co-interventions and biological-naïve participants were excluded from a systematic review as they would have induced intransitivity.2 Therefore, authors should consider, for example, the eligibility of trials of co-interventions that are known to be associated with higher efficacy compared with monotherapy. Discrepancies in the distributions of effect modifiers manifest in the data as disagreement between direct and indirect estimates, known as statistical incoherence, and can sometimes also be a source of important heterogeneity. Several statistical tests exist and should be used to check coherence, both globally (in the whole network) and locally (in parts of the network). If incoherence and/or heterogeneity is present, subgroup analyses and network meta-regression may be used to further identify the potential sources. Another key step is the evaluation of publication bias, where assessment of small-study effects constitutes an important step. This is checked visually through a modified version of the meta-analysis funnel plot called 'comparison-adjusted funnel plot'. When large asymmetries are present in the plot, small-study effects are likely acting. Network meta-regression can help to identify the causes. Additionally, sensitivity analyses should always be planned and conducted to assess if the results are robust to different methodological choices, such as the exclusion of small studies or studies at high risk of bias. A clear presentation of the findings is paramount and can be challenging to produce, especially when the network is large. The overall network effects are usually reported in forest plots while the relative effects between every combination of treatments are summarized in league tables. When many treatments are available, the number of two-by-two relative effects quickly becomes very large: for example, in the network of 20 treatments, the number of two-by-two comparisons reached 190.2 An advantage of NMA is its ability to provide a coherent ranking of treatments, for which the most popular metric is the Surface Under the Cumulative Ranking Curve (SUCRA).6 SUCRA values range between 0% and 100% (the higher the value, the higher the likelihood that the treatment is top ranked). However, it is important for these to be interpreted in conjunction with the relative effects results otherwise misleading conclusions can be made. For example, the SUCRA values of the Psoriasis Area and Severity Index 90 outcome for infliximab, secukinumab and brodalumab were 93·6, 76·2 and 68·4, respectively,2 but when comparing the two-by-two relative effects with each other, these three drugs did not show significant statistical differences in efficacy due to large uncertainty in the results. Thus, ranking measures should always be reported with the relative effects. Furthermore, several approaches have been developed to evaluate the certainty of the evidence obtained from NMAs.7, 8 CINeMA (Confidence In Network Meta-Analysis: http://cinema.ispm.ch/) is a web application extending GRADE (Grading of Recommendations Assessment, Development and Evaluation) that considers six domains to evaluate the certainty of the evidence: within-study bias, reporting bias, indirectness, imprecision, heterogeneity and incoherence. Rating the certainty of evidence with these approaches enhances the transparency, reproducibility and credibility of the results. In summary, NMAs are complex and challenging but if well conducted, they can provide the highest level of evidence in comparative effectiveness research. There is a need for collaborative work when conducting NMAs between expert clinicians, those with expertise in the conduct of systematic reviews, and methodologists and statisticians experienced in NMA. International efforts are needed to encourage authors and reviewers to follow the existing guidelines to limit the publication of poor-quality NMAs. Robin Guelimi: Conceptualization (equal); validation (equal); writing – original draft (equal); writing – review and editing (equal). Silvia Metelli: Conceptualization (equal); validation (equal); writing – original draft (equal); writing – review and editing (equal). Emilie Sbidian: Conceptualization (equal); validation (equal); writing – original draft (equal); writing – review and editing (equal). Esther J van Zuuren: Conceptualization (equal); validation (equal); writing – original draft (equal); writing – review and editing (equal). Carsten Flohr: Conceptualization (equal); validation (equal); writing – original draft (equal); writing – review and editing (equal). Jo Leonardi-Bee: Conceptualization (equal); validation (equal); writing – original draft (equal); writing – review and editing (equal). Laurence Le Cleach: Conceptualization (equal); validation (equal); writing – original draft (equal); writing – review and editing (equal).