Object defect detection based on data fusion of a 3D point cloud and 2D image

Abstract In the process of defect detection, there can be interference factors such as poor image quality and missing point cloud data as a result of the complexity of the acquisition environment. Certain limitations can be found relying solely on point cloud data processing or image feature detection. Therefore, this paper tries a more intuitive and effective exploration. Firstly, an algorithm named hole boundary points detection of point cloud, based on multi-scale principal component analysis, is proposed, which can achieve the preliminary detection of hole boundary points while calculating the normal vector of each point in the point cloud. Then the boundary contour of each hole is constructed by a polygon growth algorithm. Finally, we use the complementary information of the 3D point cloud and 2D image to explore the origin of holes and realize the ‘true’ and ‘false’ classification of holes. The experimental results show that our algorithm can successfully detect point cloud holes and can also distinguish them from object defects, providing data support for subsequent hole filling and defect measurement.