Abscisic acid (ABA) plays important roles in plant stress response, especially in drought and chilling resistance. In order to reveal the relationship between ABA and chilling tolerance in rice (Oryza sativa L.), the chilling-sensitive cultivar ‘Longjing11’ (LJ11) and chilling-tolerant cultivar ‘Longdao5’ (LD5) were pretreated with different concentrations of ABA, and then their tolerance to low temperature was analyzed. The results showed that the yield, seed-setting rate, and pollen viability were significantly decreased in the sensitive cultivar LJ11 under chilling stress; however, there was no significant change in the tolerant cultivar LD5 comparing with the untreated control. The antioxidant enzyme activity and soluble sugar content of LD5 were higher than those of LJ11, while the malondialdehyde (MDA) content and relative electricity conductivity were found lower in LD5 than those in LJ11. Spraying 20 mg L−1 ABA can effectively alleviate the chilling damage and significantly increase the seed-setting rate of LJ11 under chilling stress. The superoxide dismutase, peroxidase, and catalase activities in the leaves were obviously increased at 2–4 days under low-temperature stress, but deceased rapidly after 5-day treatment. These results showed that proper pretreatment with ABA can increase the chilling tolerance in the chilling-sensitive rice cultivar through adjusting plants physiological characteristics such as ABA content, the activities of antioxidant enzymes, soluble sugar, and MDA content.