Humans differ vastly in the confidence they assign to decisions. Although such under- and overconfidence relate to fundamental life outcomes, a computational account specifying the underlying mechanisms is currently lacking. We propose that prior beliefs in the ability to perform a task explain confidence differences across participants and tasks, despite similar performance. In two perceptual decision-making experiments, we show that manipulating prior beliefs about performance during training causally influences confidence in healthy adults ( N = 50 each; Experiment 1: 8 men, one nonbinary; Experiment 2: 5 men) during a test phase, despite unaffected objective performance. This is true when prior beliefs are induced via manipulated comparative feedback and via manipulated training-phase difficulty. Our results were accounted for within an accumulation-to-bound model, explicitly modeling prior beliefs on the basis of earlier task exposure. Decision confidence is quantified as the probability of being correct conditional on prior beliefs, causing under- or overconfidence. We provide a fundamental mechanistic insight into the computations underlying under- and overconfidence.