Hybrid Shape Deformation for Face Reconstruction in Aesthetic Orthodontics

Achieving accurate face reconstruction with geometry details from a single-view images is an important task for orthodontics. Although the 3D Morphable Model (3DMM) based methods provide an effective framework, the low-dimensional linear space is insufficient to cover geometric details. In this paper, we propose a hybrid shape deformation representation with multi-branch supervision for detail prediction. In orthodontic scenarios, shape deformation can be considered as the aggregation of intuitive appearance component and ambiguous geometry component, which is involved in frontal shape and depth correction respectively. Hence orthogonal decomposition is employed to decompose the shape deformation into frontal-plane position offset and depth offset. Frontal-plane position offset is represented in an explicit-local-dependent manner based on grid deformation while depth offset is represented in an implicit-local-dependent manner based on dense prediction. To facilitate orthodontic-based evaluation, we construct an orthodontic-specific dataset and design a novel metric to involve the relative position dependency between regions of interest. Experimentally, we demonstrate outstanding performance of face reconstruction on FaceScape, MICC Florence and orthodontic-specific dataset with both quantitative and qualitive evaluation.