Background Patients with chronic low back pain (CLBP) exhibit changes in paraspinal muscles fat infiltration and cross-sectional area (CSA). However, the relationship between lumbar disc herniation (LDH) and paraspinal muscles changes in CLBP patients remains unclear. Objective To analyze the relationship between LDH, fat infiltration in paraspinal muscles, and their CSA in patients with CLBP. Methods A total of 494 patients (201 males and 293 females) with an average age of 45.75 ± 12.93 years with CLBP were included in this study. Conventional magnetic resonance scanning sequences, inverted phase and Iterative Decomposition of water and fat with the Echo Asymmetry and Least-Square Estimation quantitation (IDEAL-IQ) sequences were used, and patients were assessed for pain on a visual analog scale (VAS). LDH was assessed based on T2 axial images at the L4-S1 level. Multifidus and erector spinae CSA and proton density fat fraction (PDFF) were measured bilaterally at the central level of the L4-S1 disc using the above-mentioned sequences and normalized by the square of height. Results Patients with LDH had a higher PDFF of the paraspinal muscles at the L4/5 level compared to those without LDH ( p < 0.05). Additionally, the VAS score of patients with LDH was significantly higher than that of the control group ( p < 0.05). However, there was no significant difference in the change of paraspinal muscles CSA between patients with CLBP with and without LDH. Conclusion In patients with LDH, there is increased fatty infiltration in the paraspinal muscles, with no significant changes in CSA. Different types of LDH are not related to paraspinal muscles fatty infiltration and CSA.