Controlling for confounding in renal injury from sevoflurane in noncardiac surgery: propensity scores versus multivariable regression. Comment on Br J Anaesth 2022; 129: 182–90

Editor—We read with great interest the study by Park and colleagues1Park M. Jung K. Cho H.S. Min J.J. Renal injury from sevoflurane in noncardiac surgery: a retrospective cohort study.Br J Anaesth. 2022; 129: 182-190Abstract Full Text Full Text PDF PubMed Scopus (4) Google Scholar who determined whether sevoflurane anaesthesia was associated with renal injury after noncardiac surgery in a retrospective cohort study. They found that exposure to sevoflurane for >3 h was not associated with increased risk of postoperative acute kidney injury (AKI), although the reported adjusted odds ratio of 1.32 (95% confidence interval: 0.99–1.76) did not exclude a clinically important increase in risk. We suggest caution when interpreting the results and conclusions of this study because of several issues with the study design and statistical analysis. Firstly, their tables 1 and 2 listing baseline and intraoperative variables showed that 156 (1.1%) of total included patients had ‘chronic renal failure’ and that 59 (0.4%) patients were in ICU with 25 mechanically ventilated before surgery. It is unclear whether patients with end-stage renal disease (who were not at risk of developing AKI) and critically ill patients who might have already developed AKI before surgery were excluded from the study cohort. In addition, the incidence of AKI (2.4%) in this study was relatively low compared with previous reports (5–13%) for noncardiac surgical patients,2Mathis M.R. Naik B.I. Freundlich R.E. et al.Preoperative risk and the association between hypotension and postoperative acute kidney injury.Anesthesiology. 2020; 132: 461-475Crossref PubMed Scopus (95) Google Scholar which is surprising as only surgeries lasting >3 h were included. The authors did not report their protocol for postoperative serum creatinine monitoring, so it is unclear whether patients with no postoperative creatinine measurement were excluded or classified as not developing AKI. Secondly, the authors used propensity score-based methods (inverse probability of treatment weighting and propensity score matching) for confounding control when estimating the association between sevoflurane exposure and postoperative AKI. Propensity score methods can be viewed as a data reduction technique, as they summarise all patient characteristics to a single covariate (the propensity score).3Schulte P.J. Mascha E.J. Propensity score methods: theory and practice for anesthesia research.Anesth Analg. 2018; 127: 1074-1084Crossref PubMed Scopus (101) Google Scholar,4Chesnaye N.C. Stel V.S. Tripepi G. et al.An introduction to inverse probability of treatment weighting in observational research.Clin Kidney J. 2021; 15: 14-20Crossref PubMed Scopus (86) Google Scholar These methods have advantages over conventional multivariable regression in studies with either a large number of confounders or a small number of events, as regression models are at significant risk of overfitting in these scenarios. However, they have important limitations compared with multivariable regression when the effective sample size is adequate. For example, propensity score matching suffers from loss of power resulting from analysing only a subset of patients, especially when the matching ratio is low.5Brazauskas R. Logan B.R. Observational studies: matching or regression?.Biol Blood Marrow Transplant. 2016; 22: 557-563Abstract Full Text Full Text PDF PubMed Scopus (67) Google Scholar In the study by Park and colleagues,1Park M. Jung K. Cho H.S. Min J.J. Renal injury from sevoflurane in noncardiac surgery: a retrospective cohort study.Br J Anaesth. 2022; 129: 182-190Abstract Full Text Full Text PDF PubMed Scopus (4) Google Scholar only 38.3% of total eligible patients was included in the 1:1 propensity score matching analysis, resulting in a relatively wide confidence interval for the effect estimate. Inverse probability weighting has some advantages over other propensity score methods as it retains data from all or nearly all study participants. However, when there is marked covariate imbalance, extreme weights at the tails of the propensity score distribution may increase variance and produce less precise estimates.4Chesnaye N.C. Stel V.S. Tripepi G. et al.An introduction to inverse probability of treatment weighting in observational research.Clin Kidney J. 2021; 15: 14-20Crossref PubMed Scopus (86) Google Scholar Other propensity score-based methods, including stratification by propensity score and using propensity score as a covariate for adjustment, were not used in the study by Park and colleagues.1Park M. Jung K. Cho H.S. Min J.J. Renal injury from sevoflurane in noncardiac surgery: a retrospective cohort study.Br J Anaesth. 2022; 129: 182-190Abstract Full Text Full Text PDF PubMed Scopus (4) Google Scholar Both methods have the merit of keeping all eligible patients in the analysis. However, they are generally not recommended as these methods do not balance covariates and reduce confounding to the same degree as propensity score matching and inverse probability weighting.3Schulte P.J. Mascha E.J. Propensity score methods: theory and practice for anesthesia research.Anesth Analg. 2018; 127: 1074-1084Crossref PubMed Scopus (101) Google Scholar,6Austin P.C. The relative ability of different propensity score methods to balance measured covariates between treated and untreated subjects in observational studies.Med Decis Making. 2009; 29: 661-677Crossref PubMed Scopus (342) Google Scholar Their list of adjustment variables (in their tables 1 and 2)1Park M. Jung K. Cho H.S. Min J.J. Renal injury from sevoflurane in noncardiac surgery: a retrospective cohort study.Br J Anaesth. 2022; 129: 182-190Abstract Full Text Full Text PDF PubMed Scopus (4) Google Scholar requires a total of 32 degrees of freedom (after removing collinear variables). As 323 events (postoperative AKI) were observed in the cohort, the rule of thumb of 10 events per predictor for logistic regression models was not violated. Multivariable logistic regression in this case could use the full dataset to produce a more precise effect estimate. In a recent study comparing propensity score methods (including matching, stratification, inverse probability weighting, and use of propensity score as a covariate) with the conventional multivariable regression method across four large cardiovascular studies, propensity score methods were not superior to multivariable regression in giving precise effect estimates.7Elze M.C. Gregson J. Baber U. Comparison of propensity score methods and covariate adjustment: evaluation in 4 cardiovascular studies.J Am Coll Cardiol. 2017; 69: 345-357Crossref PubMed Scopus (410) Google Scholar In a simulation study, analyses using propensity scores were less precise and robust than the multivariable regression estimates when there were eight or more events per predictor.8Cepeda M.S. Boston R. Farrar J.T. Strom B.L. Comparison of logistic regression versus propensity score when the number of events is low and there are multiple confounders.Am J Epidemiol. 2003; 158: 280-287Crossref PubMed Scopus (654) Google Scholar Thirdly, Park and colleagues1Park M. Jung K. Cho H.S. Min J.J. Renal injury from sevoflurane in noncardiac surgery: a retrospective cohort study.Br J Anaesth. 2022; 129: 182-190Abstract Full Text Full Text PDF PubMed Scopus (4) Google Scholar went on to investigate risk factors for postoperative AKI using multivariable logistic regression. However, this analysis was conducted using only the propensity score matched cohort which accounted for <40% of the original cohort. The model with 37 predictors but only 143 events was at high risk of overfitting. We are also concerned that multicollinearity was probably not assessed, as highly correlated variables such as ‘chronic renal failure’ and ‘preoperative creatinine >1.2 mg dl−1’ were both in the model. In addition, modelling preoperative creatinine concentration as a dichotomous, rather than continuous, variable is inappropriate, as this may cause residual confounding.9Royston P. Altman D.G. Sauerbrei W. Dichotomizing continuous predictors in multiple regression: a bad idea.Stat Med. 2006; 25: 127-141Crossref PubMed Scopus (1507) Google Scholar Moreover, the authors did not explore the possible interaction of sevoflurane use and duration of anaesthesia in the logistic regression model, which might offer further insights for answering their research question. In summary, propensity score methods are not necessarily superior to conventional multivariable regression for control of confounding in observational studies. Multivariable regression should be the technique of choice when the effective sample size is adequate. Besides the numbers of events and covariates, the regression modelling process requires careful assessment of other factors, such as avoiding categorisation of continuous variables, checking for multicollinearity, and including important interactions in the model. Propensity score methods nonetheless remain a useful alternative to control for imbalances, especially when the number of events is relatively low (e.g. seven or fewer events per predictor).8Cepeda M.S. Boston R. Farrar J.T. Strom B.L. Comparison of logistic regression versus propensity score when the number of events is low and there are multiple confounders.Am J Epidemiol. 2003; 158: 280-287Crossref PubMed Scopus (654) Google Scholar The authors declare that they have no conflicts of interest. Renal injury from sevoflurane in noncardiac surgery: a retrospective cohort studyBritish Journal of AnaesthesiaVol. 129Issue 2PreviewSevoflurane is metabolised into Compound A and fluoride that carry a hypothetical risk of nephrotoxicity. However, a clinically significant association between sevoflurane use and acute kidney injury (AKI) in humans has not been established. Full-Text PDF Open Archive