Compared with solid-colored fabrics, the textures in yarn-dyed fabric images are more complex, making the task of defect detection more challenging. To achieve efficient detection, this study proposes an automatic detection framework for dyed fabric defects. The proposed framework consists of a hardware system and a detection algorithm. For efficient and high-quality acquisition of fabric images, an image acquisition assembly equipped with three sets of light sources and a mirror was developed. In addition, a defect detection algorithm based on Fourier convolution and a convolutional autoencoder is proposed. Abandoning the common way of adding noise, this paper proposes to generate image pairs for training using a random masking method in the training phase. In the autoencoder, some traditional convolutional layers are replaced with Fourier convolutional layers. Ablation experiments verify the effectiveness of the mask generation method and Fourier convolution. Compared with other defect detection methods, the proposed method achieves the best performance, which verifies the superiority of the method. The maximum detection speed of the developed system can reach 41 meters per minute, which can meet real-time requirements.