Introduction: The impact of colorectal surgery on older adults’ (≥65 years) health-related quality of life (HRQoL) is rarely reported. Methods: This prospective multi-institutional cohort study involving 217 participants investigated factors affecting older colorectal cancer (CRC) patients’ HRQoL following surgery with the EORTC QLQ-C30 and the HADS. Results: The trend of the global QoL and individual subscales show an overall dip in QoL from baseline to pre-discharge, before slowly rising until the 6-month mark. Baseline score of individual outcome measures was the only constant predictor of the outcome measure over time. Conclusion: Age was not found to be a significant predictor of HRQoL in older adults undergoing colorectal surgery, while baseline scores of individual outcome measures consistently predicted post-operative HRQoL scores. Interventions should be targeted at specific subgroups of older adults scheduled for colorectal surgery, such as those with lower baseline scores.