Attention-Based Policy Distillation for UAV Simultaneous Target Tracking and Obstacle Avoidance

Nowadays, deep reinforcement learning (DRL) has made remarkable achievements in the research of unmanned aerial vehicle (UAV) applications. However, much of the current research on UAVs focuses on a single task, which limits its applicability in many scenarios. Therefore, we have conducted research on UAV target tracking and obstacle avoidance tasks. The DRL-based solutions integrate the objectives of multi-tasks into a common reward function making the training unstable and difficult to converge, resulting in the loss of objectives or collisions. To that end, we propose a novel target following and obstacle avoidance solution based on policy distillation of the task attention mechanism. First, we train the two tasks of UAV target following and UAV obstacle avoidance respectively. Both networks are trained using Dueling Double Deep Q Network to learn the corresponding policy in an end-to-end manner. Then we extract the two policies that have been trained separately into a memory buffer. Meanwhile, we can perceive the collision risk through the state of the current environment of the UAV to assign the weights of the two tasks in the attention mechanism. Therefore, our method can adaptively focus on the corresponding tasks according to the current state. We conducted simulation experiments using the Virtual Robot Experimentation Platform. Our study presents compelling experimental findings: (1) Our novel approach outperforms state-of-the-art methods by achieving superior tracking accuracy and extended tracking durations across diverse environments, all while mitigating collision incidents. (2) The distilled policy we have developed exhibits robust generalization capabilities when applied to previously unencountered environments.