Enhancement of noisy and compressed videos by optical flow and non-local denoising

A new method for denoising video sequences is presented. The method is able to deal with poor quality sequences affected by both noise and compression, as for example mobile phone videos recorded in low-light conditions. In this real scenario, the usual white Gaussian uniform noise assumption fails, and the state-of-the art denoising algorithms are ineffective. The proposed method first estimates a signal dependent noise model at each level of a multi-scale pyramid; then, a variance stabilization transform is applied at each scale; and, finally, a denoising algorithm is applied on the stabilized image sequence. This algorithm takes advantage of self similarity and redundancy of adjacent frames and uses motion compensation by regularized optical flow methods, which permits robust patch comparison in a spatio-temporal volume. The experiments illustrate that the proposed method is able to correctly remove highly correlated noise from real dark and compressed movie sequences. Finally, in order to assess numerically the performance of the proposed algorithm, we introduce a new database of noisy and compressed image sequences with known ground truth. The numerical comparisons with the introduced database corroborate the superior performance of the method.