Uncertainties still exist about the effect of vegetarian or vegan diets on glycemic and cardiometabolic risk factors in individuals with type 2 diabetes mellitus (T2DM), although plant-based diets are thought to be beneficial for cardiometabolic health. The aim was to investigate whether vegetarian or vegan diets can improve blood glucose and cardiometabolic health in patients with T2DM compared with omnivorous diets. Five databases (PubMed, Web of Science, Cochrane Library, Scopus, and Embase) were searched for eligible randomized controlled trials (RCTs) up to May 24, 2024. Two authors independently performed the data extraction and quality assessment. Nine RCTs (681 participants) were included in this meta-analysis. The results indicated that vegetarian or vegan diets could reduce glycosylated hemoglobin, type A1C (HbA1c) (weighted mean difference [WMD] = -0.36%; 95% CI: -0.54, -0.19; P < .001), low-density-lipoprotein cholesterol (WMD = -0.16 mmol/L; 95% CI: -0.26, -0.07; P = .001, and body mass index (WMD = -0.94 kg/m2; 95% CI: -1.43, -0.45; P = .0002) in a population with T2DM; however, they resulted in no significant improvement in systolic blood pressure and fasting plasma glucose. In subgroup analyses, the positive effects of a vegan diet were superior to a vegetarian diet for cardiometabolic health. HbA1c was reduced when vegetarian or vegan intake was more than 12 weeks. In conclusion, vegetarian or vegan diets could be utilized as a synergistic intervention in the T2DM population, contributing to cardiovascular disease prevention. In the future, the proportion of components that make up a plant-based diet should be explored. PROSPERO registration no. CRD42024578613.