Stripe noise removing remains a challenging inverse problem for researchers in the field of infrared image. In this paper, an infrared image stripe noise removing method is proposed based on least squares and gradient domain guided filtering. Firstly, least squares embedded with bilateral filter is applied to smooth infrared image and extract high frequency image information. Then, a one-dimensional gradient domain guided filtering, which provides more efficient and precise vertical stripe information, is proposed to separate detailed texture information and vertical stripe noise in high-frequency images. Finally, the denoised image is obtained by subtracting the noisy image from the separated stripe noise image. Experimental results on public datasets reveal that compared with state-of-the-art methods the proposed method can receive satisfactory results in computational efficiency, denoising effect and detail feature preservation.