A Multi-defect detection system for sewer pipelines based on StyleGAN-SDM and fusion CNN

• A multi-defect detection system for sewer pipelines based on StyleGAN v2 and fusion CNN is proposed. • A multi-defeat image generation model, called StyleGAN-SDM, is proposed by integrating StyleGAN v2 and sharpness discrimination model (SDM) to generate multi-defeat images and automatically select clear images. • A multi-defect classification model (MDCM) based on fusion CNN, which combines the Inception network architecture and the Residual network architecture, is proposed to classify the on-site images into four categories. • On-site video detection for multiple defects is realized by the computer vision library of OpenCV. With the development of deep learning, convolutional neural networks (CNN) have been gradually used in pipeline defeats detection. However, due to the complex environment inside the pipeline, few defeat images are not enough for the training of CNN. A multi-defect detection system based on StyleGAN-SDM and fusion CNN for sewer pipelines is proposed in this paper. First, aiming at the problem of data acquisition and small data volume, raw images are preprocessed by StyleGAN-SDM, which integrates StyleGAN v2 and sharpness discrimination model (SDM) to generate multi-defect images and automatically select clear images. The indexes of Inception-Residual score (IRS), accuracy and macro-F1 score to evaluate the quality of the images generated are 2.968 ± 0.024, 99.64%, and 0.997, respectively. Second, to improve the detection accuracy, a multi-defect classification model (MDCM) based on fusion CNN, which combines Inception network and Residual network, is proposed to classify the on-site images into four categories. Third, compared with conventional deep-learning methods, the mean accuracy and macro-F1 score of the proposed model reach 95.64% and 0.955, which are increased by 1.51% and 0.015 by StyleGAN-SDM, respectively. Finally, to solve the timeliness problem of on-site detection, a real-time multi-defeat detection system for sewer pipelines is established with the computer vision library of OpenCV. Some on-site videos are detected with the mean speed of 24.11 FPS and these results could aid the staff.