Abstract The effects of different land‐use regimes on C sequestration in the terrestrial ecosystems were investigated in the Loess Plateau. We found that the surface soil (0–10 cm) was more active at sequestering carbon after land had been used as grassland. When cropland was converted to grassland, C storage in surface soil (0–10 cm) increased. Grazing exclusion led to vegetation and soil restoration, thus increasing the sequestration of atmospheric C. Intensive cultivation led to a decline of soil organic carbon content in the surface soil of cropland. Long‐term application of organic manure accompanied with moderate chemical fertiliser can greatly enhance the C storage in the soil layer below the tilled layer in cropland. In this region, both grassland and cropland can contribute to C sequestration when proper management practice is implemented.