Human Knowledge Enhanced Reinforcement Learning for Mandatory Lane-Change of Autonomous Vehicles in Congested Traffic

Mandatory lane-change scenarios are often challenging for autonomous vehicles in complex environments. In this paper, a human-knowledge-enhanced reinforcement learning (RL) method for lane-change decision making is proposed, where the human intelligence is integrated with RL algorithm in a multiple manner. First, this paper constructs a complex ramp-off scenario with congested traffic flow to help agents master lane-change skills. On the basis of the Twin Delayed Deep Deterministic policy gradient (TD3) algorithm, the human prior experience is encoded into reward function and safety constraints offline, and the online guidance of experts is also introduced into the framework, which can limit the unsafe exploration during the training process and provide demonstration in complex scenarios. The experimental results indicate that our method can effectively improve the training efficiency and outperform typical RL method and expert drivers, without specific requirements on the expertise. The proposed method can enhance the learning ability of RL based driving strategies.