Soil is the largest organic carbon pool in the terrestrial biosphere, and a small variation of soil organic carbon (SOC) can substantially affect the global carbon cycle and climate. Climate change is a major driver affecting the dynamics of SOC; however, our understanding about the responses of SOC in alpine ecosystems to climate change is quite limited. In particular, the differences of SOC dynamics at different depths were rarely reported. In this study, we investigated the impacts of precipitation variations and warming on SOC dynamics at both top and deep soils in the Qilian Mountains in Northwestern China using a machine learning approach and climate sensitivity experiments. Our simulation revealed the temporal inconsistency between topsoil SOC (in the top 20 cm, denoted as SOC20) and deeper soil SOC dynamics—SOC20 showed a decreasing trend since 2009 which is earlier than that (2012) in the top 100 cm soil (SOC100). We also found that SOC100 may be more sensitive to warming due to the strengthened microbial decomposition rate and additional carbon source through deepened active layer. On the contrary, SOC20 presented more intense responses to precipitation than SOC100, which was mainly attributed to the different responses of upland and lowland SOC to precipitation variations. Our projection indicated that SOC20 may not substantially change under future climate trajectories because the enriched SOC induced by increased precipitation may offset the carbon loss via warming. However, SOC100 was projected to decrease significantly due to the enhanced carbon emissions via warming induced strengthened decomposition rate, additional carbon source from the deepened active layer, and exposed soil carbon to the atmosphere caused by ground subsidence and disrupted soil horizons resulting from thawed frozen soil. In brief, this study deepened our understanding of the mechanism of climate effect on SOC dynamics and can be helpful for regional soil ecological security assessment and risk projection.