Confidence of evidence should be considered in ranking of treatments in the network meta-analysis

In their network meta-analysis, Alessandro Rodolico and colleagues1Rodolico A Bighelli I Avanzato C et al.Family interventions for relapse prevention in schizophrenia: a systematic review and network meta-analysis.Lancet Psychiatry. 2022; 9: 211-221Google Scholar explored 11 models of family interventions for patients with schizophrenia. Treatments were ranked by probability of being the best (P-scores) on the basis of prevention of relapse at 12 months (the primary outcome). Based on the P-scores, the authors found that, when compared with treatment as usual, family psychoeducation was the most effective intervention in preventing relapse at 12 months (odds ratio 0·18, 95% CI 0·12–0·27), followed by systemic-oriented family intervention (0·33, 0·17–0·66), and family psychoeducation combined with family behavioural or skills training and mutual skills training support (0·35, 0·23–0·53), and concluded that family psychoeducation alone was superior to the other combined models. However, when considering the limitations of P-scores, this conclusion might be inappropriate. It could be very misleading to rank the effectiveness or harms of treatments by relying on the P-scores only and ignoring the confidence or quality of evidence, because P-scores focus on point estimates of effect and do not consider the possibility that chance could explain differences between P-scores, the magnitude of the absolute difference between rankings, and, most importantly, the confidence of evidence.2Mbuagbaw L Rochwerg B Jaeschke R et al.Approaches to interpreting and choosing the best treatments in network meta-analyses.Syst Rev. 2017; 6: 79Google Scholar, 3Rücker G Schwarzer G Ranking treatments in frequentist network meta-analysis works without resampling methods.BMC Med Res Methodol. 2015; 15: 58Google Scholar For example, in the league table of the primary outcome of relapse in the Article by Rodolico and colleagues,1Rodolico A Bighelli I Avanzato C et al.Family interventions for relapse prevention in schizophrenia: a systematic review and network meta-analysis.Lancet Psychiatry. 2022; 9: 211-221Google Scholar although family psychoeducation was associated with the most effective estimate, the confidence of evidence was very low, which means there is very low certainty that the high effect of family psychoeducation is true. Therefore, the evidence for that intervention is untrustworthy. Instead, the systemic-oriented family intervention appears to be most compelling, as the treatment was associated with a large effect and moderate confidence of evidence. Although the authors used the Confidence in Network Meta-Analysis (CINeMA) framework4Nikolakopoulou A Higgins JPT Papakonstantinou T et al.CINeMA: an approach for assessing confidence in the results of a network meta-analysis.PLoS Med. 2020; 17e1003082Google Scholar to assess the confidence of estimates, the confidence of evidence was not considered in the ranking process. Therefore, we suggest the authors consider the confidence of evidence and use the partially contextualised framework5Brignardello-Petersen R Izcovich A Rochwerg B et al.GRADE approach to drawing conclusions from a network meta-analysis using a partially contextualised framework.BMJ. 2020; 371m3907Google Scholar to rank the treatments, which could avoid the limitations of P-scores. If the authors use the new approach, they might get more appropriate, informative, and user-friendly conclusions. We declare no competing interests. Family interventions for relapse prevention in schizophrenia: a systematic review and network meta-analysisAlmost all family intervention models were efficacious in preventing relapse in schizophrenia. Family psychoeducation alone, without behavioural or skills training, was superior to the more complex models. Our results suggest that in contexts where there are financial constraints, family psychoeducation alone should be implemented. Full-Text PDF Confidence of evidence should be considered in ranking of treatments in the network meta-analysis – Authors' replyWe thank Shitong Xie and colleagues for commenting on our network meta-analysis that investigated family interventions for patients with schizophrenia.1 Full-Text PDF