The mental representation of brief temporal durations, when assessed in standard laboratory conditions, is highly accurate. Here we show that adding or subtracting temporal durations systematically results in strong and opposite biases, namely over-estimation for addition and under-estimation for subtraction. The difference with respect to a baseline temporal reproduction task changed across durations in an operation-specific way and survived correcting for the effect due to operation sign alone, indexing a reliable signature of arithmetic processing on time representation. A second experiment replicated these findings with a different set of stimuli. This novel behavioral marker conceptually mirrors in the time domain the representational momentum found with motion, whereby the estimated spatial position of a visual target is displaced in the direction of motion itself. This momentum effect in temporal arithmetic suggests a striking analogy between time processing and visuospatial processing, which might index the presence of common computational principles.