GraphLight: Graph-based Reinforcement Learning for Traffic Signal Control

Adaptive traffic signal control can alleviate traffic congestion and improve throughput. A decentralized multi-agent reinforcement learning (MARL) is a promising data-driven method for learning a cooperative traffic signal policy for dynamic traffic networks. Although most existing MARL algorithms employ neural networks, e.g., convolutional neural network (CNN), to capture the low-dimensional feature embedding from the high-dimensional state space, the features of dynamic traffic network states cannot be effectively exacted by CNN s due to some features of objects ignored, e.g., the directions of vehicles. Based on the above observations, we propose a decentralized graph-based multi-agent advantage actor-critic method, referred to as GraphLight, to implement traffic signal control at multiple intersections. In the proposed GraphLight, the graph convolutional network is employed to extract features of dynamic traffic networks, the states of neighbor agents are used to learn cooperative control policies. We numerically evaluate the proposed GraphLight in an environment with 25 intersections. The experimental results show that the proposed method outperforms the state-of-the-art methods in terms of multiple metrics, and can adapt better the dynamic traffic networks.