Training a Reinforcement Learning Agent with AutoRL for Traffic Signal Control

Traffic signal control system affects the efficiency of transportation. Adaptive traffic signal control has attracted attention for automatically adjusting the phase according to different traffic conditions. Researchers have been trying to apply deep reinforcement learning to the design of adaptive traffic signal control. The meta-parameter like $\gamma$ , the discount rate of future reward, is crucial in reinforcement learning. Researchers, especially across disciplines, need to conduct extensive experiments to find appropriate values for the meta-parameters. Automated Reinforcement Learning (AutoRL) can automate the design choices of meta-parameters instead of manual tuning. The gradient-based meta-learning algorithm belongs to AutoRL. To save time in exploring meta-parameters, we integrate the gradient-based meta-learning algorithm into DQN (GBML-DQN). We conduct experiments on the traffic simulator SUMO. Our results show that GBML-DQN promotes convergence of the Q-value function and avoids overestimation to some extent, especially in the case of inappropriate training settings, whereas DQN fails in training.