3D-Trajectory and Phase-Shift Design for RIS-Assisted UAV Systems Using Deep Reinforcement Learning

Unmanned aerial vehicle (UAV) can effectively work as temporary base station or access point in the air to transfer/receive data to/from ground terminals (GTs). However, UAV-GT links might be blocked by ground obstacles, like buildings in urban area, leading to a poor performance on data transferring rate. To address this problem, reconfigurable intelligent surface (RIS), as a promising technique, can intelligently reflect the received signals between UAV and GT to significantly enhance the communication quality. Under this deployment of RIS-assisted UAV, we intend to jointly optimize the 3D-space of the UAV and the phase-shift of the RIS to maximize the data transferring rate of the UAV, while minimizing the UAV propulsion energy. The joint problem is non-convex in its original form and difficult to be timely solved by using traditional method, like successive convex approximation (SCA). Therefore, to facilitate the online decision making to this joint problem, we leverage deep reinforcement learning (DRL) to learn the near-optimal solution, and the well known Double Deep Q-Network (DDQN) and Deep Deterministic Policy Gradient (DDPG) algorithms are ultilized. Numerical results show that DRL can effectively improve the energy-efficiency performance of the RIS-Assisted UAV system, compared with benchmark solutions.