Abstract Background Phytoestrogens (PEs) are naturally estrogen-like chemicals, and increasing evidences have indicated their endocrine disruption effects on male reproductivity, but the conclusions from previous epidemiological studies were controversial. Objective To examine the associations between nine phytoestrogens in semen and semen quality in a Chinese population. Methods In this cross-sectional study, a total of 1319 reproductive-aged men were recruited from Shenzhen, China. Semen phytoestrogens were measured by ultra-performance liquid chromatography and tandem mass spectrometry. Semen quality was assessed by sperm concentration, sperm count, progressive motility, total motility, volume, and the sperm motion parameters. Both multivariate linear regression and logistic regression models were conducted to evaluate the associations between semen phytoestrogens and semen quality with adjustment for confounders. Results In logistic regression models, we found significant associations between semen secoisolariciresinol (SEC) and lower sperm concentrations (odd ratios (OR): 2.38; 95% confidence interval, 95% CI: 1.47, 3.93), sperm counts (OR: 2.27; 95% CI: 1.34, 3.94), and total motility (OR: 1.55; 95% CI: 1.08, 2.24). Negative associations were also observed for semen genistein (GEN) with sperm counts (OR: 2.28; 95% CI: 1.29, 4.14; p for trend = 0.04) and sperm concentrations (OR: 1.98; 95% CI: 1.21, 3.03; p for trend = 0.07). Semen naringenin (NAR) were found to be positively associated with progressive motility (OR: 0.57; 95% CI: 0.38, 0.83) and total motility (OR: 0.57; 95% CI: 0.40, 0.81). Results from multivariate linear regression models were similar to those from logistic regression models for semen SEC, GEN, and NAR. Conclusions We suggested that semen levels of phytoestrogens may be associated with semen quality in men. Further investigations are warranted to confirm the findings in prospective studies and to explore the underlying mechanism.