Test-Retest Reliability in Metric Conjoint Experiments: A New Workflow to Evaluate Confidence in Model Results

Metric conjoint studies are a popular research design in the entrepreneurship domain. For these studies, test-retest reliabilities of ρ > .70 or higher are an often-cited reliability criterion. Despite their widespread use, however, there is little rigorous analysis of whether test-retest reliability in metric conjoint studies relates to model efficacy. Informed by a systematic literature review, we conducted two Monte Carlo simulations to evaluate the effects of various determinants of test-retest reliability in conjoint experiments. We then illustrate a workflow for entrepreneurship researchers employing conjoint designs to better evaluate—and communicate—confidence in statistical models estimated from conjoint data.