Graph filtering plays a pivotal role in the domain of graph signal processing. In this paper, we propose a novel algorithm for designing autoregressive moving average (ARMA) graph filters. Our method is built upon classical least squares (LS) techniques. A reciprocal polynomial is introduced, making the resulting problem more tractable. To address stability concerns, a second-order factorization of the denominator polynomial is further incorporated. We develop an alternating optimization approach to tackle the design problem, where numerator coefficients, coefficients of the denominator’s second-order factors (SOCs), and those of its reciprocal polynomial are updated alternately. Simulation results demonstrate that our proposed algorithm outperforms existing graph filter design methods in terms of design accuracy.