Reinforcement Learning Based Dual-UAV Trajectory Optimization for Secure Communication

Unmanned aerial vehicles (UAV) can serve as aerial base stations for users due to their flexibility, low cost, and other characteristics. However, due to the high flight position of UAVs, the air-to-ground (ATG) channels usually dominate with line-of-sight (LoS), which can be easily eavesdropped by multiple eavesdroppers. This poses a challenge to secure communication between UAVs and ground users. In this paper, we study a UAV-aided secure communication in an urban scenario where a legitimate UAV Alice transmits confidential information to a legitimate user Bob on the ground in the presence of several eavesdroppers around it and a UAV Jammer sends artificial noise to interfere with the eavesdroppers. We aim to maximize the physical layer secrecy rates in the system by jointly optimizing the trajectories of UAVs and their transmitting power. Considering the time-varying characteristics of channels, this problem is modeled as a Markov decision process (MDP). An improved algorithm based on double-DQN is proposed in the paper to solve this MDP problem. Simulation results show that the proposed algorithm can converge quickly under different environments, and the UAV transmitter and UAV jammers can find the optimal location correctly to maximize the information secrecy rate. It also shows that the double-DQN (DDQN) based algorithm works better than the Q-learning and deep Q-learning network (DQN).