Distributed Node Classification with Graph Attention Networks

Developing scalable machine learning solutions that meet the demand of real-world applications is crucially important in domains such as cybersecurity, smart grids, and social networks. Despite the rising interest in the analysis of graph data using big data frameworks, existing works are usually limited to the adoption of conventional machine learning models, due to their simplicity and availability as off-the-shelf algorithms in popular libraries. As a result, resorting to more sophisticated models such as deep neural networks for graph analysis in a fully distributed learning setting is still an open challenge. In this paper, we propose a distributed workflow for node classification in graphs. We focus on graph attention networks, and devise a distributed model training approach leveraging the Apache Spark, GraphX, and Horovod frameworks. Our workflow consists of feature engineering, graph partitioning, model deployment, and model training stages, which take place in a fully distributed manner. Experimental results show that leveraging graph partitioning is a feasible strategy for distributed model training on multiple workers equipped with GPUs. Specifi-cally, the randomized and informed graph partitioning strategies analyzed in our experiments present satisfactory results in terms of both accuracy and scalability with two benchmark graph datasets for node classification.