Three-dimensional leaf edge reconstruction using a combination of two- and three-dimensional phenotyping approaches

Abstract Background: The physiological functions of plants are carried out by leaves, which are important organs. The morphological traits of leaves serve multiple functional requirements and demands of plants. Traditional techniques for quantifying leaf morphology rely largely on two-dimensional (2D) methods, resulting in a limited understanding of the three-dimensional (3D) functionalities of leaves. Notably, recent advancements in surveying technologies have improved 3D data acquisition processes. However, there are still challenges in producing accurate 3D-representations of leaf morphologies, particularly leaf edges. Therefore, in this study, we propose a method for reconstructing 3D leaf edges using a combination of 2D image instance segmentation and curve-based 3D reconstruction. Results: The proposed method reconstructed 3D leaf edges from multi-view images based on deep neural network-based instance segmentation for 2D edge detection, SfM for estimating camera positions and orientations, leaf correspondence identification for matching leaves among multi-view images, curve-based 3D reconstruction for estimating leaf edges as 3D curve fragments, and B-spline curve fitting for integrating curve fragments into a 3D leaf edge. The method was demonstrated on both virtual and actual plant leaves. On the virtually generated leaves, we evaluated the accuracy of the 3D reconstruction by calculating standardized Fréchet distance, which reveals that small leaves and high camera noise pose greater challenges to reconstruction. To balance the number and precision of 3D curve fragments, we proposed guidelines for setting the threshold for how only reliable curve fragments are reconstructed based on simulated data. These guidelines suggested that the threshold becomes lower with greater occlusions, larger leaf size, and camera positional error greater than a certain level. We also found the number of images does not affect the optimal threshold except in very few cases. Moreover, the proposed method succeeded in reconstructing holes in the leaf when the number of holes is three or less. Conclusions: In this study, a nondestructive method for 3D leaf edge reconstruction was developed to address the 3D morphological properties of plants, which have been challenging to evaluate quantitatively. It is a promising way to capture whole plant architecture by combining 2D and 3D phenotyping approaches adapted to the target anatomical structures.