Edge-Aware Spatial Propagation Network for Multi-view Depth Estimation

Deep learning has made great improvements in multi-view stereo. Recent approaches typically adopt raw images as input and estimate depth through deep networks. However, as a primary geometric cue, edge information, which captures the structures of scenes well, is ignored by the existing multi-view stereo networks. To this end, we present an Edge-aware Spatial Propagation Network, named ESPDepth, a novel depth estimation network that utilizes edges to assist in the understanding of scene structures. To be exact, we first generate a coarse initial depth map with a shallow network. Then we design an Edge Information Encoding (EIE) module, to encode edge-aware features from the initial depth. Subsequently, we apply the proposed Edge-Aware spatial Propagation (EAP) module, to guide the iterative propagation on cost volumes. Finally, the edge optimized cost volumes are utilized to obtain the final depth map, serving as a refinement process. By introducing the edge information in the propagation of cost volumes, the proposed method performs well when capturing geometric shapes, thus alleviating the negative effects of the greatly changed depth on edges of real scenes. Experiments on ScanNet and 7-Scenes datasets demonstrate our method produces precise depth estimation, gaining improvements both on global structures and detailed regions.