The objective of this study was to investigate the aging depth gradient distribution of high viscosity modified asphalt (HVMA) under the effect of solar radiation and diffuse oxygen. The HVMA samples with different thicknesses were prepared to perform the weathering aging under different solar radiation intensities. Then, the Fourier transform infrared spectroscopy test, gel permeation chromatography test, and fluorescence microscopy test were used to study the chemical composition and micromorphology characteristic of HVMA at different aging depths. Afterward, the dynamic shear rheology test and multiple stress creep recovery test were applied to investigate the macroscopic rheological performance of HVMA. The test results showed that the effect of solar radiation aging diffused from the asphalt surface to the asphalt inside gradually. As the solar radiation intensity and aging time increased, the significant aging depth of HVMA increased continuously. At the surface, HVMA absorbed the solar light near the characteristic absorption band to activate the asphalt molecules and polymer molecules and had oxidation and depolymerization reactions with the full participation of diffuse oxygen. Simultaneously, HVMA also showed continuous microcracks at the surface due to the direct radiation of solar light. As the aging depth increased, the cracking phenomenon disappeared, and the asphalt oxidation rate and polymer degradation rate of HVMA decreased gradually. The increase of solar radiation intensity causes more severe aging at the surface and a faster downward propagation rate of aging effect, thus resulting in accelerated deterioration of the macroscopic rheological performance of HVMA.