Packet Routing Against Network Congestion: A Deep Multi-agent Reinforcement Learning Approach

The continuous growth of the network data would lead to the increased network congestion and the throughput decline. In this paper, we investigate the packet routing problem based on deep multi-agent reinforcement learning, where each router chooses the next hop router by itself intelligently. We design the modified deep Q-network in each router to evaluate the neighbor routers. The routers, each acting as an agent, choose the next hop router based on their local observation. Then they transfer the packets to the chosen routers and receive the reward and the observation of the next hop routers. Using their experience, the routers learn to improve the packet routing strategy by updating their Q-networks. We demonstrate that with proper reward set and training mechanism, the routers in the network can work in a distributed way to reduce the computational complexity compared with the single-agent reinforcement learning based algorithm. And the proposed algorithm can further reduce the congestion probability and improve the network performance.