Regularizing inverse problems in image processing with a manifold-based model of overlapping patches

Local patch-based models have been shown to be effective in numerous image processing applications and have become the core of the state-of-the-art denoising, inpainting and structural editing algorithms. Most such modeling approaches mainly rely on searching for similar patches in the set of available patches. However, the apparent similarity between sufficiently small (e.g., 5×5 pixels) image regions motivates modeling them with a low-dimensional manifold instead and suggests the existence of a simple parametrization for it. Although there exist manifold models for a single patch, it has remained an open problem how to efficiently represent an entire image in terms of its overlapping patches drawn from the underlying non-linear manifold. We propose to consider an image to lie on the intersection of separate manifolds corresponding to different overlapping patches, which we approximate with affine subspaces in a kernel-induced feature space. In contrast to our previous work on this topic, here we solve the intersection and preimage problems simultaneously, ensuring the existence of a suitable solution in the input space. This significantly improves the performance and robustness of our method. Our method incorporates any desired equality constraints on the image, and thus can be used to regularize any linear inverse problem with the manifold intersection model. Our experimental results show nearly perfect compressive sensing reconstruction of images whose patches are well described by a manifold model, as well as exceptional performance in denoising and inpainting.