FloorUSG: Indoor floorplan reconstruction by unifying 2D semantics and 3D geometry

This paper proposes a multistage floorplan reconstruction approach from RGB images and a dense 3D mesh, called FloorUSG, by combining 2D plane instances and 3D plane primitives. In primitive detection, plane instances inferred from images complement the results of traditional 3D plane detection well. And in the optimization, both the plane confidence and the geometric quality of data are considered to select the optimal subset from the candidates. Different from existing methods that rely on delicate corner detection from a planar graph or pure geometric 3D plane detection, our framework accurately recovers the location of the floorplan via 2D–3D primitive fusion. Experimental results indicate that our method has the ability to recover detailed structures of scenes of different scales and can reconstruct the floorplan from imperfect data with high robustness compared to the state-of-the-art algorithms.