Understanding the alterations to starch multi-scale structure induced by ultrasound treatment can help in determining the effective application of ultrasound in functional-starch preparation. This study aimed to comprehensively characterize and understand the morphological, shell, lamellae, and molecular structures of pea starch granules treated by ultrasound under different temperatures. Scanning electron microscopy and X-ray diffraction analyses showed that UT (ultrasound treatment) did not change C-type of crystalline, but caused a pitted surface and endowed a looser structure and higher enzyme susceptibility as the temperature increased above 35 °C for pea starch granules. Fourier transform infrared spectroscopy and small-angle X-ray scattering analyses revealed that UT reduced the short-range ordering and increased the thickness of semi-crystalline and amorphous lamellae by inducing starch chain depolymerization, which was manifested by molecule weight and chain length distribution analysis. The sample ultrasound-treated at 45 °C had the higher proportion of B2 chains compared with the other ultrasound-treated samples because the higher ultrasonic temperature altered the disruption sites of starch chains.