Detection of loosening angle for mark bolted joints with computer vision and geometric imaging

Mark bars drawn on the surfaces of bolted joints are widely used to indicate the severity of loosening. The automatic and accurate determination of the loosening angle of mark bolted joints is a challenging issue that has not been investigated previously. This determination will release workers from heavy workloads. This study proposes an automated method for detecting the loosening angle of mark bolted joints by integrating computer vision and geometric imaging theory. This novel method contained three integrated modules. The first module used a Keypoint Regional Convolutional Neural Network (Keypoint-RCNN)-based deep learning algorithm to detect five keypoints and locate the region of interest (RoI). The second module recognised the mark ellipse and mark points using the transformation of the five detected keypoints and several image processing technologies such as dilation and expansion algorithms, a skeleton algorithm, and the least square method. In the last module, according to the geometric imaging theory, we derived a precise expression to calculate the loosening angle using the information for the mark points and mark ellipse. In lab-scale and real-scale environments, the average relative detection error was only 3.5%. This indicated that our method could accurately calculate the loosening angles of marked bolted joints even when the images were captured from an arbitrary view. In the future, some segmentation algorithms based on deep learning, distortion correction, accurate angle and length measuring instruments, and advanced transformation methods can be applied to further improve detection accuracy.