Introduction Phosphodiesterase type 5-inhibitors (PDE5is) are the first-line treatment for patients with diabetes mellitus-induced erectile dysfunction (DMED), however, some patients are non-responser to PDE5is. We performed a perspective, randomized, comparative study to explore the efficacy of low intensity extracorporeal shock wave treatment (Li-ESWT) combined with vacuum erectile device (VED) in the treatment of DMED patients who were unresponsive to PDE5is. Methods One hundred and five eligible patients were randomly divided into three groups: group A (VED), group B (Li-ESWT) and group C (VED plus Li-ESWT). Follow-up was conducted at 4 weeks, 8 weeks and 12 weeks after the end of treatment. The erectile function was estimated by the international index of erectile function-erectile function domain (IIEF-EF), erection hardness score (EHS), sexual encounter profile questions 2 and 3 (SEP2 and SEP3) and global assessment question 1 and 2 (GAQ1 and GAQ2) before and after treatment. The changes of five points in IIEF-EF were calculated as the minimal clinical important difference (MCID), which was considered as the main index of efficacy. Results The MCID was achieved in 14.7%, 14.7% and 17.6% patients in group A at the follow up on 4 weeks, 8 weeks and 12 weeks, respectively (36.4%, 39.4% and 36.4% in group B; 36.4%, 51.5%, and 66.7% in group C). There were significant differences in the percentage of MCID cases between group A and group C at the follow up on 12 weeks ( P <0.001), as well as that between group B and group C ( P =0.014). Additionally, comparison in MCID within group C showed that there were significant differences between 4 weeks and 12 weeks follow-up ( P =0.014). Conclusion Our findings indicated the combined therapy Li-ESWT and VED was more beneficial to shift turn PDE5is non-responders to responders for moderate patients with DMED than VED or Li-ESWT monotherapy. Moreover, this study provided evidence that patients with DMED who failed after taking oral PDE5is could attempt to opt for an alternative physicotherapy (Li-ESWT or VED) prior to more invasive alternatives.