A lowlight image enhancement method learning from both paired and unpaired data by adversarial training

Abstract In recent years, deep learning has been widely used in the field of lowlight image enhancement. In this paper, we propose a novel deep-learning-based lowlight image enhancement method, which could learn from both paired and unpaired data, and perform end-to-end contrast enhancement and noise reduction simultaneously. Concretely, we first employ a multi-level content loss on paired synthesized data for training enhancer to recover detail better. Then, we propose a GAN-based domain adaptation mechanism, which applies an adversarial training strategy and a novel gamma-correction-based self-supervised content loss on abundant unpaired real data, for training the enhancer to perform better on real lowlight images. Furthermore, a Patch-GAN-based noise reduction mechanism is proposed to adversarially train the enhancer to better reduce noise in real lowlight images. Finally, we improve the enhancer by introduce attention mechanism and global feature to original U-net, make it more suitable for lowlight image enhancement task. We conduct experiments on several common datasets and the results show that our method outperforms other state-of-the-arts under a variety of image quality assessment metrics. And when applied as pre-processing module, our method can improve the classification accuracy on lowlight dataset by 1.7%, outperforming other methods too.