Automatic augmentation and segmentation system for three-dimensional point cloud of pavement potholes by fusion convolution and transformer

The regular three-dimensional (3D) detection of potholes is essential for the assessment of pavement conditions. However, some problems associated with the segmentation of pavement potholes, such as the scarcity of 3D point cloud data of pavement potholes and the segmentation interference of asphalt aggregate particle gaps ('False potholes'), need to be solved. Therefore, a 3D point cloud augmentation and segmentation system for pavement potholes was proposed in this paper. The system comprises a fusion 3D convolution with transformer for 3DGAN (Conv-Trans-3DGAN) and a fusion 3D convolution and transformer 3D Segmentation network called Conv-Trans-3DSeg. First, a Conv-Trans-3DGAN was developed for pavement potholes, based on the improved GAN by fusion convolution and transformer, to generate 3D point cloud data. Conv-Trans-3DGAN was used to generate high-quality 3D point cloud data for pavement potholes. Second, an innovative network of Conv-Trans-3DSeg was developed to segment 3D point cloud data for pavement potholes. The global and local features of the pothole region were extracted by Conv-Trans-3DSeg to solve the problem of asphalt aggregate gaps leading to false potholes. The segmentation accuracy and F1-score of a trained Conv-Trans-3DSeg model can reach 92.075% and 91.688% respectively. Ablation comparison experiments were conducted, and the accuracy and F1-score were improved by 4.733% and 3.03%, respectively. Compared with the better known PointNet, PointNet++, Point Transformer (PT) and PointNeXt, the accuracy of the proposed model was improved by 7.694%, 2.819%, 3.34% and 3.743% respectively; the F1 score was improved by 5.211%, 2.721%, 2.102% and 3.894% respectively. In addition, the proposed Conv-Trans-3DSeg has a segmentation efficiency of 21.3FPS. The results show that the proposed segmentation network achieves accurate and efficient recognition and segmentation of three-dimensional point cloud data of potholes.