Detection method for table grape ears and stems based on a far-close-range combined vision system and hand-eye-coordinated picking test

• A set of far-close-range stereoscopic vision system was constructed to detect grape ear and grape stem. • The identification and localization algorithms of grape centroid and point-cutting of grape stems were proposed. • The calculation model of the point-cutting of grape stems was established. The point-cutting localization had the success rate of 92% on sunny day, 82% on sunny backlight and 86% on overcast. • The close-range vision system may add positioning compensation to the binocular vision system. • The grape-picking robot can complete the table grape-picking task within an average time of 53.4 s. To improve the picking success rate of table grapes in the process of automatic picking, it is important to accurately locate the point picking of grape ears and the cutting point of grape stems. In this research, a set of far-close-range stereoscopic vision systems was constructed to detect grape ears and grape stems. First, identification algorithms for grape ear centroid were proposed. A series of image operations were performed to complete the recognition and feature extraction of far-range grapes, including median filtering, threshold segmentation, morphological operation, executable region marking, and the extraction of the target centroid. Second, a calculation model of the cutting point of grape stems was established. The selected region of interest (ROI) of the close-range grape stem was determined to carry out edge detection and use a cumulative probability Hough transform. The straight line detection and cutting point positioning of the close-range grape stem were completed. Finally, based on LabVIEW software, the measurement and control system of the grape-picking robot was developed. Experiments verified that the identification of grape stems and localization algorithm of the cutting point had higher reliability, with success rates of 92% on sunny days, 82% during sunny backlight and 86% for overcast conditions. The grape-picking tests showed that the grape-picking robot can complete the table grape-picking task within an average time of 53.4 s under the given computer configuration.