Pre-training local and non-local geographical influences with contrastive learning

Geographical influences fundamentally help to improve the performance of location-based services (LBS). However, existing LBS approaches rely on abundant task-specific labeled data in an end-to-end manner, which often causes overfitting and sparsity problems. One effective way is to pre-train contextualized representations on unlabeled data with self-supervision to capture intrinsic correlations between locations and their contexts. In this paper, we propose a novel local and non-local geographic representation (LNGR) model with contrastive self-supervised learning, which is able to simultaneously incorporate geospatial proximity as a local geographical influence and relative distance differences as a non-local geographical influence. To capture the inherent dependency between the geographical influences, we pre-train sequential (for non-local) and surrounding (for local) contextual encoders in a unified framework with three different types of self-supervised objectives, hence promoting the quality of contextual point-of-interest (POI) representations. We evaluate our pre-trained model for next POI recommendation on six check-in datasets. The extensive experimental results demonstrate that the superiority of LNGR over existing pre-training and end-to-end recommendation methods. Besides, we further show the effective robustness and generalization ability of our pre-trained model when task-specific labeled data is scarce.