Self-Augmented Heterogeneous Face Recognition

Heterogeneous face recognition (HFR) is quite challenging due to the large discrepancy introduced by cross-domain face images. The limited number of paired face images results in a severe overfitting problem in existing methods. To tackle this issue, we proposes a novel self-augmentation method named Mixed Adversarial Examples and Logits Replay (MAELR). Concretely, we first generate adversarial examples, and mix them with clean examples in an interpolating way for data augmentation. Simultaneously, we extend the definition of the adversarial examples according to cross-domain problems. Benefiting from this extension, we can reduce domain discrepancy to extract domain-invariant features. We further propose a diversity preserving loss via logits replay, which effectively uses the discriminative features obtained on the large-scale VIS dataset. In this way, we improve the feature diversity that can not be obtained from mixed adversarial examples methods. Extensive experiments demonstrate that our method alleviates the over-fitting problem, thus significantly improving the recognition performance of HFR.