Stepwise Logistic Regression

To the Editor: Through simulations, Pace1 demonstrates in an editorial the difficulties of stepwise automatic variable selection as applied to logistic regression. I agree with Dr. Pace that one needs to exercise caution with any kind of model selection technique and that prior knowledge in the area of study is extremely important in covariate selection. In his editorial, Pace refers to three variants of automatic variable selection: forward selection, backward elimination, and stepwise regression and the simulations were presented for model selection using stepwise regression. From his supplementary data analysis codes, the model selection technique employed in the simulation was backward elimination. (For his specification of the model the software chooses backward elimination as the default method). Using the backward elimination method, there were 825 instances in the 1000 simulations with at least one significant covariate at P < 0.05. When the same simulations were repeated with either forward selection or stepwise regression, no covariate was found significant at P < 0.05. Pace also uses the Akaike Information Criterion2 to choose the model in backward elimination. The Bayesian Information Criterion,2 provides a greater penalty for the addition of an extra covariate. Simulation using the Bayesian Information Criterion failed to choose a single covariate at P < 0.05 in each of the 1000 simulations. Furthermore, with Pace’s simulation (i.e., backward elimination using Akaike Information Criterion for model selection), if one used multiple hypothesis tests3 and computed adjusted P values, then the number of instances where a significant covariate was selected would decrease to 414 instances (332 with 1, 73 with 2, and 9 with 3 covariates) (see our simulations in the online appendix, available at www.anesthesia-analgesia.org). These results demonstrate the effects the various model selection techniques and selection criteria have on the selection of the final model. Therefore, to interpret the data properly, it is always advisable to consider various model building and selection strategies in statistical analysis, as suggested by Dr. Pace. Srikesh G. Arunajadai, PhD Department of Anesthesiology and Biostatistics Columbia University New York City, New York [email protected]