Sensitivity of Goodness of Fit Indexes to Lack of Measurement Invariance

Abstract Two Monte Carlo studies were conducted to examine the sensitivity of goodness of fit indexes to lack of measurement invariance at 3 commonly tested levels: factor loadings, intercepts, and residual variances. Standardized root mean square residual (SRMR) appears to be more sensitive to lack of invariance in factor loadings than in intercepts or residual variances. Comparative fit index (CFI) and root mean square error of approximation (RMSEA) appear to be equally sensitive to all 3 types of lack of invariance. The most intriguing finding is that changes in fit statistics are affected by the interaction between the pattern of invariance and the proportion of invariant items: when the pattern of lack of invariance is uniform, the relation is nonmonotonic, whereas when the pattern of lack of invariance is mixed, the relation is monotonic. Unequal sample sizes affect changes across all 3 levels of invariance: Changes are bigger when sample sizes are equal rather than when they are unequal. Cutoff points for testing invariance at different levels are recommended. ACKNOWLEDGMENTS I would like to thank Kristopher Preacher and Donna Coffman for their thoughtful comments on an earlier version of this article. I also express appreciation to Stephen West and Roger Millsap for their insights on measurement invariance. I am grateful to the Quantitative Forum in the Psychology Department at the University of North Carolina at Chapel Hill for fruitful discussions at the early stages of this work.