EyesGAN: Synthesize human face from human eyes

Face recognition recently has achieved remarkable success in many fields, especially in mobile payment, authentication, criminal investigation, and city management. However, face occlusion is still the key problem in person identification, such as in the field of anti-terrorism, criminal cases and public security. To solve this problem, an improved end-to-end deep generative adversarial network (named EyesGAN) has been proposed to synthesize human face from human eyes in this paper, which can be used as a potential scheme for masked face recognition. BicycleGAN is chosen as the baseline and effective improvements have been achieved. First, the self-attentional mechanism is introduced so that the improved model can more effectively learn about the internal mapping between human eyes and face. Second, the perceptual loss is applied to guide the model cyclic training and help with updating the network parameters so that the synthesized face can be of higher-similarity to the ground truth face. Third, EyesGAN has been designed by getting the utmost out of the performance of the perceptual loss and the self-attentional mechanism in GANs. A dataset of eyes-to-face synthesis has been reconstructed based on the public face datasets for training and testing. Finally, the faces synthesized by EyesGAN have been quantitatively and qualitatively compared with the results of existing methods. Extensive experiments demenstrate that our proposed method has performed better than the state-of-the-art methods in terms of Average Euclidean Distance, Average Cosine Similarity, Synthesis Accuracy Percentage, Fréchet Inception Distance, and Baidu face recognition rate (the accuracy achieved 96.1% on 615 test data of CelebA database). In this paper, the feasibility of synthesizing human face from human eyes has been explored, and the attention map shows that our network can predict other parts of the face from eyes.