An automatic defect classification and segmentation method on three-dimensional point clouds for sewer pipes

With the development of deep learning (DL), sewer pipe inspection on two-dimensional (2D) images has achieved remarkable accuracy. However, extracting defect measurements from these 2D images is challenging due to the curved nature of pipes and the lack of depth information. Point clouds can restore the three-dimensional (3D) information of objects. To effectively identify defects in disordered and sparse point clouds, a 3D sewer pipe classification and segmentation method was proposed. In the encoder, the original point clouds are sampled and grouped and the local features in the clusters are extracted by two symmetric functions (1 × 1 convolution and the maximization function) to process the points with permutation invariance. In the decoder, the multi-scaling abstract features are upsampled using feature pyramid network (FPN) to predict the category of each point. Especially, the network structure and training strategy of the inspection method is optimized to improve the inspection accuracy. Furthermore, two data augmentation methods, namely random scaling and point jitter, are used to increase the data volume. An ablation experiment shows that the optimization of network structure can effectively improve the performance of the inspection model and the novel training strategies can stabilize the training process and prevent overfitting. Comparison among the state-of-the-art networks demonstrates that the proposed segmentation model attains the highest mIoU of 94.15 %, which is improved by 11.46 % with the optimization of network structure and training strategy. For the classification task, the F1 score and accuracy of the established model are 6.79 % and 5.46 % higher than PointNet++, respectively. These results signify the high-accuracy defect inspection capability of our proposed method on 3D point clouds of sewer pipelines.