A UAV Path Planning Method in Three-Dimensional Urban Airspace based on Safe Reinforcement Learning

Under the demand of urban terminal "Last Mile Delivery" scenario, finding a safe and efficient UAV path planning method is a crucial issue of current research. Nowadays, reinforcement learning is widely used in UAV path planning, but it is difficult to ensure the safety of the learning or execution phases due to the lack of hard constraints. Aiming at the constraints above, this paper studies how to combine safety properties with RL algorithm to find a safe path and proposes a safe reinforcement learning method called Shield-DDPG for UAV path planning. In the method, a protection mechanism Shield is mainly introduced to prevent the algorithm from outputting unsafe actions. Further, the state space, action space, and reward function are specifically improved for efficiency and safety. Then we compare the Shield-DDPG algorithm with the DDPG and RRT algorithm in some different scenarios, and the results show that the proposed algorithm has a better performance. With the proposed path planning method, UAV can learn well to efficiently and safely reach the destination via calling the trained policy. This research is of great importance to UAV operations and practical applications in complex urban airspace.