The relationship between nitrogen and phosphorus concentrations in rivers and lakes and their influencing factors have been global concerns. However, how air temperature changes affect the nitrogen and phosphorus concentrations in rivers and lakes remains unknown. In this study, we conducted analyses linking the characteristics of air temperature to monthly nitrogen and phosphorus monitoring datasets of the lake and adjacent inflowing rivers in the Lake Chaohu basin from 2014 to 2018. We found that the variations in the mean air temperature of the antecedent 7 days significantly affected the nitrogen and phosphorus concentrations in shallow eutrophic lakes, and the air temperature threshold ranged from 3 to 27 °C. As the air temperature increased, the nitrogen concentrations in Lake Chaohu decreased, but the phosphorus concentrations showed an upwards trend. The lake had a “buffer” function, as indicated by the more pronounced response of nitrogen and phosphorus to increasing air temperature in western Lake Chaohu (TN = − 0.085 mg L−1 °C−1, TP = + 0.004 mg L−1 °C−1) than eastern Lake Chaohu (TN = − 0.034 mg L−1 °C−1, TP = + 0.003 mg L−1 °C−1). The decreasing trend of nitrogen concentrations (average declining rate: 0.090 mg L−1 °C−1) in inflowing rivers with increasing air temperature was even more pronounced than that in Lake Chaohu. However, no significant statistical relationship was found between the phosphorus concentrations and air temperature in most inflowing rivers. Therefore, this study emphasizes the need to further unravel the coupling mechanism between internal nutrient loads and climate factors while reducing external nutrient loads.