Objective The objective was to investigate the frequency and type of chromosomal abnormalities and Y chromosome microdeletions in infertile men with azoospermia and oligozoospermia to ensure appropriate genetic counseling before assisted reproduction in Eastern China. Methods A total of 201 infertile men (148 with azoospermia and 53 with oligozoospermia) were enrolled. Real-time PCR using six Y-specific sequence-tagged sites of the azoospermia factor (AZF) region was performed to screen for microdeletions. Karyotype analyses were performed on peripheral blood lymphocytes with standard G-banding. Results Out of 201 infertile patients, 22 (10.95%) had Y microdeletions [17/148 (11.49%) men with azoospermia and 5/53 (9.43%) men with oligozoospermia]. The most frequent microdeletions were in the AZFc region, followed by the AZFa+b + c, AZFb+c, AZFa, and AZFb regions. Chromosomal abnormalities were detected in 18.91% (38/201) of patients, 34 of which were sex chromosome abnormalities (16.92%) and 4 of which were autosomal abnormalities (1.99%). Chromosomal abnormalities were more prevalent in men with azoospermia (22.97%) than in those with oligozoospermia (7.55%). Conclusions We detected a high incidence of chromosomal abnormalities and Y chromosomal microdeletions in infertile Chinese men with azoospermia and oligozoospermia. These findings suggest the need for genetic testing before the use of assisted reproduction techniques.