A Hierarchical Reinforcement Learning Algorithm Based on Attention Mechanism for UAV Autonomous Navigation

Unmanned Aerial Vehicles (UAVs) are increasingly being used in many challenging and diversified applications. Meanwhile, UAV’s ability of autonomous navigation and obstacle avoidance becomes more and more critical. This paper focuses on filling up the gap between deep reinforcement learning (DRL) theory and practical application by involving attention mechanism and hierarchical mechanism to solve some severe problems encountered in the practical application of DRL. More specifically, in order to improve the robustness of DRL, we use averaged estimation function instead of the normal value estimation function. Then, we design a recurrent network and a temporal attention mechanism to improve the performance of the algorithm. Third, we propose a hierarchical framework to improve its performance on long-term tasks. Some realistic simulation environments, as well as the real-world, are used to evaluate the proposed UAV autonomous navigation method. The results demonstrate that our DRL-based navigation method performs well in different environments and outperforms the original DrQ algorithm.