Domain Embedding Transfer for Unequal RGB-D Image Recognition

Most recent unsupervised domain adaptation (UDA) approaches concentrate on single RGB source to single RGB target task. They have to face the real-world scenario, where the source domain can be collected from multiple modalities, e.g., RGB data and depth data. Our work focuses on a more practical and challenging scenario which recognizes RGB images by learning from RGB-D data under the label inequality scenario. We are confronted with three challenges: multiple modalities in the source domain, domain shifting problem and unequal label numbers. To address the aforementioned settings, a novel method, referred to as Domain depth Embedding Transfer (DdET) is proposed, which takes advantage of the depth data in the source domain and handles the domain distribution mismatch under label inequality scenario simultaneously. We conduct comprehensive experiments on five cross domain image classification tasks and observe that DdET can perform favorably against state-of-the-art methods, especially under label inequality scenario.