<i>Multiple Object Tracking-by-Detection for Fruit Counting on an Apple Tree Canopy</i>

Abstract. Accurate estimation of the number of fruit on apple trees is a potentially valuable tool for enabling growers to better manage their operations. In particular, the fruit count information can be used for harvest planning, sales forecasting, and optimization of crop load management. A method for counting fruit on apple trees using RGB video sequences was implemented using a deep learning object detector based on the Faster R-CNN architecture and optical flow for object tracking. The detection and tracking mechanisms are integrated to count unique fruit detections across continuous image sequences. The proposed methodology increases overall detection accuracy by minimizing counting errors due to occluded and clustered fruit. For fruit detection in still images, a precision of 92% and recall of 82% are reported. The largest source of error came from heavily occluded fruits, which comprised 55% of fruit in the dataset and had a detection accuracy of 73%. The high percentage of heavily occluded fruit motivated the development of the video tracking algorithm, which increased the overall detection rate to 97% across the tested video sequences – effectively minimizing the occlusion problem by taking advantages of multiple viewpoints to detect partially occluded or hidden fruit. Total fruit counts for these sequences had an average error of 10% due to the introduction of false positives during video sequence analysis.