A YOLOv3-based computer vision system for identification of tea buds and the picking point

Famous tea industry which need to harvest tea buds has great economic benefits. However, the harvesting is time-consuming and labor-intensive, especially with the shortage of labor currently, an intelligent tea bud picking robot is urgently needed. The vision system is a precursor to the development of a tea bud picking robot. To resolve such issues, we applied robotics and deep learning technologies to develop a computer vision system for intelligent picking of tea buds. The system was designed to recognize tea buds and extract their picking points. A method for locating the picking points was proposed based on a combination of YOLO-v3 algorithm, semantic segmentation algorithm, skeleton extraction and minimum bounding rectangle. An intelligent tea end-effector based on Personal Computer and microcontroller collaborative control was designed to solve the picking problem like complex shading and easy breakage. Thus, the picking rate of the overall system was improved. Based on Openmv smart camera embedded mobilenet_v2 algorithm as the visual model of the classification device, so that the quality of tea buds was preliminatively classified. Finally, the effects of different shooting angles and shooting methods as well as the accuracy of target detection and semantic segmentation algorithms on the extraction of tea bud picking points were investigated. The results show that the average accuracy of YOLO-v3 for identification of tea buds is 71.96% and the average horizontal positioning error of the robotic arm is 2.4 mm. Also, the average depth positioning error is 4.2 mm and the accuracy of tea bud picking point extraction is 83%. After the test, the successful picking rate of tea buds is 80% by this computer vision system of robot. The results of this study is potential to develop a machine-based tea picking system for industry and would contribute to the development of precision agriculture.