Visual identification and pose estimation algorithms of nut tightening robot system

To realize the automatic tightening of nuts, a nut tightening robot system strategy based on machine vision was designed in this work. The strategy was based on three stages: nut image calibration, nut identification, and nut pose estimation. In the first stage, the template pose image of the nut and the coordinates of the nut center in this nut image were obtained by calibration. In the second stage, a nut identification algorithm based on improved the backbone feature extraction network and area generation network of Faster-RCNN was presented, which improved the efficiency and accuracy of nut identification. In the last stage, a nut pose estimation algorithm based on Fourier and log-polar coordinate transformation was presented to solve the rotation angle, translation, and scale of the nut relative to the template nut image and then obtain the rotation angle of the sleeve and the central coordinate of the nut. An experimental nut tightening robot platform was also set up in this work. The results of 50 tests showed that the proposed detection methods could identify nuts with 100% accuracy, and with the proposed pose estimation methods, the average error of the rotation angle of the nut was 0.057°, and the average error of the center position of the nut in $$x$$ and $$y$$ directions was ± 0.05 mm and of $$z$$ direction was ± 0.5 mm. The experimental results showed that the nut tightening robot scheme and algorithm designed in this work were feasible in nut identification and pose estimation and met the requirements of insertion accuracy in the process of nut tightening.