Autonomous Driving Strategy for Highway Parallel-type On-ramp Merging Based on Deep Reinforcement Learning

About 30% of all traffic accidents in China are caused by highway merging areas, particularly near the on-ramp area. This paper proposes an autonomous driving strategy for merging at the parallel-type on-ramp, which is centered on the deep reinforcement learning approach Deep Q-Network (DQN) and can simultaneously consider the safety, efficiency and comfort of vehicles during the merging process. This paper retrieves 834 merging events from the human driving trajectory dataset CitySim, in contrast to prior studies that used simulation. The DQN model receives as inputs the speed and coordinates of the onramp vehicles in the merging area, as well as the vehicles nearby in front of and behind it on the main road, and outputs the lateral and longitudinal accelerations of the on-ramp vehicles. The total reward of the DQN model consists of three components, namely merging safety reward, merging efficiency reward and merging comfort reward. The model training results are compared with human driving data, and the results show that the strategy proposed in this paper can make on-ramp vehicles perform safe, efficient and comfortable merging behaviour, which can provide a useful reference for the implementation of autonomous driving in on-ramp merging areas.