Modeling Causal Irrelevance in Evaluations of Configurational Comparative Methods

In volume 44 of Sociological Methodology, Lucas and Szatrowski argued that the method of qualitative comparative analysis (QCA) suffers from a built-in confirmation bias due to a proclivity for including conditions in its output models whose corresponding factors are not systematically correlated with the endogenous factor. The authors therefore urged that QCA be abandoned. With this comment, we pursue four related objectives: first, we explain why correlation-based evaluation designs for testing QCA’s power of discrimination with respect to causally irrelevant factors are unsuitable; second, we show how appropriate tests must be constructed; third, we offer an R function that implements a routine for such tests; and fourth and finally, we conduct three series of comprehensive tests, all of whose results indicate that QCA does not suffer from the kind of confirmation bias criticized by Lucas and Szatrowski.