I propose that temporal integration is ubiquitous in visual perception, because it serves an adaptive role. To support this idea, I draw together evidence from historically separated research fields that target different timescales. At one extreme, this concerns the detection and discrimination of successive stimuli within intervals of less than a quarter of a second. At an intermediate level, associated with attentional episodes, intervals between half a second up to a few seconds are considered. Finally, at the other extreme, this involves high-level, conceptual events across intervals of multiple seconds or even minutes. Across such varying intervals, the nature of temporal integration and the resultant perceptual events are clearly different. I nevertheless propose that temporal integration should be understood as a continuous process that serves a common adaptive goal: To maximize the amount of useful information, at minimal costs, tailored to the observer's current needs and circumstances. Emerging from this viewpoint are several research directions that might be pursued on the topic of temporal integration, and on its consequences for perception and memory.