Objectives Diabetes mellitus (DM) is associated with risk of sarcopenia. However, whether glycated hemoglobin (HbA1c) levels are associated with sarcopenia has not reached a consensus. The purpose of this study was to examine the association between diabetes status based on HbA1c levels and sarcopenia indices in community-dwelling older adults stratified by body mass index (BMI) status. Design An observational cross-sectional study. Setting and Participants General community setting. The participants were 18,940 community-dwelling older adults (mean age 73.7 ± 5.7 years, 45.0% men). Methods According to their medical history for diabetes mellitus and blood HbA1c levels the participants were classified into the control group (no DM history and HbA1c ≤5.6%), pre-DM group (no DM history and HbA1c 5.7%–6.4%), or DM group (DM history or HbA1c ≥6.5%). For assessments of sarcopenia, muscle mass and muscle strength were measured based on the clinical definition. Results A multiple logistic regression analysis showed that the DM group did not have a clear association with sarcopenia [odds ratio (OR) 1.06, 95% confidence interval (CI) 0.82–1.38] or low muscle mass (OR 1.15, 95% CI 0.94–1.39), and was associated with low muscle strength (OR 1.21, 95% CI 1.08–1.36) in the adjusted model including BMI. There was a positive correlation between BMI and skeletal muscle index (r = 0.57, P < .001). Stratified by BMI, the DM group was associated with low muscle mass among the underweight participants (OR 1.87, 95% CI 1.03–3.39), and low muscle strength among the overweight participants (OR 1.22, 95% CI 1.04–1.43). Conclusions and Implications Diabetes status was associated with low muscle mass and low muscle strength, and the association depended on BMI. Our findings could be helpful for identifying older adults with a high risk of sarcopenia, although further study is needed.