Learning-Based Robust and Secure Transmission for Reconfigurable Intelligent Surface Aided Millimeter Wave UAV Communications

In this letter, we study the robust and secure transmission in the millimeter-wave (mmWave) unmanned aerial vehicle (UAV) communication assisted by a reconfigurable intelligent surface (RIS) under imperfect channel state information (CSI). Specifically, the active beamforming of the UAV, the coefficients of the RIS elements and the UAV trajectory are jointly designed to maximize the sum secrecy rate of all legitimate users in the presence of multiple eavesdroppers. However, the CSI is coupled with the UAV trajectory, which results in complex constraints. Furthermore, the time-related issue caused by the outdated CSI also makes the formulated problem intractable to solve. To tackle these challenges, by leveraging the deep deterministic policy gradient (DDPG) framework, a novel and effective twin-DDPG deep reinforcement learning (TDDRL) algorithm is proposed. Simulation results demonstrate the effectiveness and robustness of the proposed algorithm, and the RIS can significantly improve the sum secrecy rate.