Depth image restoration based on uncertainty-guided priority optimization

Depth images captured by commercial red green blue-depth (RGB-D) cameras commonly suffer from missing regions, which severely hinders their applicability. To effectively restore missing depth values, especially for large holes, we propose a depth image restoration based on uncertainty metrics (DRUM) for hole-filling. Specifically, pseudo-background and information entropy are proposed to quantify the uncertainty for determining the filling priority and identifying pixels needing restoration, which effectively mitigates error accumulation. Subsequently, color and gradient clues are fully exploited to guide depth value prediction, remarkably enhancing accuracy. Then, ablation studies validate the efficacy of the proposed innovations. Comparative experiments on public benchmarks demonstrate DRUM's superior performance over prevailing methods on diverse hole patterns. Finally, a fast implementation called Fast-DRUM is devised through multi-threading and selective updating. Comparative evaluations of processing time demonstrate its high efficiency. We provide new insights into reliable depth completion for practical applications.