Propranolol is the first-line treatment for infantile hemangiomas (IH). Cases of propranolol-resistant infantile hemangiomas are rarely reported. The purpose of our study was to investigate the predictive factors for poor response to propranolol. A prospective analytical study was conducted between January 2014 and January 2022 including all patients with IH who received oral propranolol therapy at a dose of 2–3 mg/kg/day maintained for at least 6 months. A total of 135 patients with IH were treated with oral propranolol. Poor response was reported in 18 (13.4%) of the patients: 72% were girls and 28% were boys. Overall, 84% of the IH were mixed, and hemangiomas were multiple in three cases (16%), nasal tip hemangiomas accounted for four cases (22%), and 15 patients (83%) had segmental hemangiomas. There was no significant association between the age or sex of the children and type of response to treatment (p > 0.05). No significant association was found between the type of hemangioma and the therapeutic outcome as well as the recurrence after treatment discontinuation (p > 0.05). Multivariate logistic regression analysis revealed that nasal tip hemangiomas, multiple hemangiomas, and segmental hemangiomas were at greater risk of poor response to beta-blockers (p < 0.05). Poor response to propranolol therapy has rarely been reported in the literature. In our series, it was approximately 13.4%. To our knowledge, no previous publications have focused on the predictive factors of poor response to beta-blockers. However, the reported risk factors for recurrence are discontinuation of treatment before 12 months of age, mixed or deep type IH, and female gender. In our study, the predictive factors for poor response were multiple type IH, segmental type IH, and location on the nasal tip.