A unified probabilistic framework of robust and efficient color consistency correction for multiple images

The task of color consistency correction for multiple images mainly arises from applications like orthoimage producing, panoramic image stitching and 3D reconstruction. In these applications, images usually have been geometrically aligned. So correspondences can be easily extracted and used to solve color correction models. Almost all previous methods assume that the color residuals of correspondences follow Gaussian distribution and solve color models based on least squares. However, correspondences often contain unreliable ones due to altered areas and misalignments, which results in unusual large color residuals, namely, outliers. Imposing color consistency constaints on unreliable correspondences significantly affects the performance of color correction since Gaussian is highly sensitive to outliers. In this paper, to solve this problem theoretically, we first propose a unified probabilistic framework that formulates global color correction as a maximum posteriori probability (MAP) estimation. It is flexible enough to allow for any assumptions of residual distribution. And most color correction methods can be explained in this unified framework. Then, to robust against outliers, we use t-distribution with heavier tails than Gaussian to fit the color residuals. It is more robust because higher probabilities can be assigned to outliers. We show that the MAP formulation based on t-distribution actually leads to weighted least squares, which downweights outliers adaptively. Besides, our framework requires no user-defined robustness parameter. Because all parameters of color models and t-distribution are optimized jointly. In addition, to decrease the huge computational cost of large scale dataset, we extend the proposed framework to a parallel vesion which can achieve efficiency and global optimal at the same time. In the experiments, we compare our approach with the state-of-the-art approaches of Shen et al., Xia et al., etc. on several challenging datasets with outliers. The results demonstrate that our approach achieves the best robustness (average color consistency scores CD=5.4, DeltaE2000=5.7 and PSNR=24.0) and the best efficiency (given 100 images, non-parallel/parallel runs more than 5/50 times faster than others). The implementation is available at https://github.com/yinxuanLi/ColorConsistencyCorrectionForMultipleImages.