Autonomous Driving Based on Approximate Safe Action

Safety limits the application of traditional reinforcement learning (RL) methods to autonomous driving. To address the challenge of safe exploration in autonomous driving tasks, a novel safe RL method called Twin Delayed Deep Deterministic Policy Gradient based on Approximate Safe Action (TD3-ASA) is proposed in this paper. In TD3-ASA, the action output by the current policy during the exploration process is modified to obtain an approximate safe action, and then the approximate safe action is utilized to train a safe policy for deployment. TD3-ASA offers several advantages: 1) TD3-ASA is sample efficient and does not need any prior knowledge; 2) TD3-ASA enhances safety both during training and deployment; 3) TD3-ASA introduces an adjustable safety correction factor that enables a tradeoff between exploration and safety. Experimental results conducted on both the MetaDrive and SpeedLimit autonomous driving test platforms demonstrate the effectiveness of TD3-ASA. TD3-ASA exhibits more than triple safety during training on MetaDrive compared to the current state-of-the-art RL method, achieving a high success rate and low deployment risk.