With a fundamentally modified structural illumination algorithm, the recently proposed speckle-illuminated Fourier ptychography can be a promising superresolution imaging technique with a large field of view. However, its imaging performance, including image resolution and signal-to-noise ratio, has been discussed less, limiting its further applications. Thus, an in-depth study of this new imaging technique is highly required. In this paper, with theoretical analysis, numerical simulations, and experiments, the influence of both diffuser roughness in the experimental setup and numerical aperture size in iterative reconstruction on the imaging performance of speckle-illuminated Fourier ptychography was studied in detail, and the result explained why a rougher diffuser and larger reconstruction aperture can generate a higher-resolution image with more noise and showed how to get optimized diffuser roughness and reconstruction aperture size by considering the trade-off between imaging resolution and signal-to-noise ratio. This work may be a good reference for high-quality imaging using speckle-illuminated Fourier ptychography.