Explanatory Cognitive Diagnostic Modeling Incorporating Response Times

Abstract This study proposes explanatory cognitive diagnostic model (CDM) jointly incorporating responses and response times (RTs) with the inclusion of item covariates related to both item responses and RTs. The joint modeling of item responses and RTs intends to provide more information for cognitive diagnosis while item covariates can be used to predict item parameters when item calibration is not feasible in diagnostic assessments or item parameter estimation errors could be too large due to small sample sizes for calibration. In addition, the inclusion of the item covariates allows the evaluation of cognitive theories underlying the test design in item development. Model parameter estimation is explored using the Bayesian Markov chain Monte Carlo (MCMC) method. A Monte Carlo simulation study is conducted to examine the parameter recovery of the proposed model under different simulated conditions in comparison to alternative competing models. Further, the application of the proposed model is illustrated using the Programme for International Student Assessment (PISA) 2012 problem‐solving items modeling both item response and RT data. The study results indicate that model parameters can be well recovered using the MCMC algorithm and the explanatory CDM jointly incorporating item responses and RTs with item covariates holds promising applications in digital‐based diagnostic assessments.