High completeness multi-view stereo for dense reconstruction of large-scale urban scenes

Multi-View Stereo (MVS) algorithms remain a significant challenge in reconstructing a 3D model with high completeness due to the difficulty in recovering weakly textured regions and detailed parts of large-scale urban scenes. Although the Image Pyramid Structure is a popular approach for dealing with weakly textured regions, it also leads to the loss of detailed information. The proposed method solves these problems with three new strategies: (1) We propose the optical flow consistency for recovering details. The optical flow consistency improved the sensitivity of the image pyramid structure to details by estimating the motion vector of each pixel point. We proposed a novel detail restorer based on optical flow consistency which improves the link between adjacent scales in the image pyramid structure. (2) Geometric consistency based on epipolar line constraints is proposed to recover weakly textured regions. The proposed epipolar line constraints improve the robustness of traditional geometric consistency, which avoids the problem of mismatching in weakly textured regions. (3) A depth-filling strategy is utilized to fill the loss of depth information of weakly textured regions. Image gradient is utilized to fill the gap of depth information. The filled result is utilized as the priori information to smooth the depth of weakly textured regions. Experimental results on the ETH3D, UDD5 and SenseFly benchmark datasets demonstrate that the proposed method outperforms three state-of-the-art methods (ACMMP, EPNet, DeepC-MVS), significantly improving the completeness of the 3D models. The source code of the develop method is available at https://github.com/Liaoyongjian1/HC-MVS.git.