RAFNet: Interdomain Representation Alignment and Fine-Tuning for Image Series Classification

Classification of remote sensing image series which differ in quality and details, has impportant implications for the analysis of land cover, whereas it is expensive and time-consuming as a result of manual annotations. Fortunately, domain adaptation (DA) provides an outstanding solution to the problem. However, information loss while aligning two distributions often exists in traditional DA methods, which impacts the effect of classification with DA. To alleviate this issue, an inter-domain representation alignment and fine-tuning based network (RAFNet) is proposed for image series classification. Inter-domain representation alignment, which is fulfilled by a variational auto-encoder (VAE) trained by both source and target data, encourages reducing the discrepancy between the two marginal distributions of different domains and simultaneously preserving more data properties. As a result, RAFNet, which fuses the multi-scale aligned representations, performs classification task in the target domain after well trained with supervised learning in the source domain. Specifically, the multi-scale aligned representations of RAFNet is acquired by duplicating the frozen encoder of VAE. Then, an information based loss function is designed to fine-tune RAFNet, in which both the unchanged and changed information implied in change maps is completely used to learn the discriminative features better and make the model more generalized for the target domain. Finally, experiment studies on three datasets validate the effectiveness of RAFNet with considerable segmentation accuracy even the target data has no access to any annotated information.