Graph-Based Cooperation Multi-Agent Reinforcement Learning for Intelligent Traffic Signal Control

In the trend of continuously advancing urban intelligent transport construction, traditional traffic signal control (TSC) struggles to make effective decisions with complex traffic conditions. Although multi-agent deep reinforcement learning (MARL) shows promise in optimizing traffic flow, most existing studies ignore the complex relationships between signal lights and fail to communicate with neighbors effectively. Moreover, the deterministic strategies generated by Q-learning-based methods struggle to be extended to large-scale urban road networks. Therefore, this paper proposes a multi-agent graph-based soft actor-critic (MAGSAC) approach for TSC, which combines graph neural networks with the Soft Actor-Critic (SAC) algorithm and extends it to multi-agent environments to address the TSC problem. Specifically, we employ graph-based networks and attention mechanism to expand the receptive domain of agents, enable environmental information to be shared among agents, and utilize the attention mechanism to filter out unimportant information. The algorithm adheres to the Centralized Training Decentralized Execution (CTDE) paradigm to minimize the non-stationarity of MARL. Finally, a rigorous experimental evaluation was conducted using the CityFlow simulator on both synthetic traffic grids and real-world urban road networks. Experimental results show that MAGSAC outperforms other TSC methods in performance metrics, including average queue length and waiting time, and achieves excellent performance under complex urban traffic conditions.