Unmanned-Surface-Vehicle-Aided Maritime Data Collection Using Deep Reinforcement Learning

Employing unmanned surface vehicles (USVs) as marine data collectors is promising for large-scale environment sensing in remote ocean monitoring network. In this article, we consider a USV-aided marine data collection network, where a USV collects data from multiple monitoring terminals while avoiding collisions with monitoring terminals and obstacles. Aiming at minimizing energy consumption and data loss, we formulate a trajectory optimization problem with practical constraints, including collision avoidance, steering angle, and velocity limitation. The problem is intractable due to the stochastic arrived data and the random emergence and movement of dynamic obstacles. To efficiently solve it, we transform it as a constrained Markov decision process (MDP) problem and address it using a target-oriented double deep ${Q}$ -learning network (D2QN)-based collision avoidance and trajectory planning algorithm. In the proposed algorithm, the USV acts as an agent to explore and learn its trajectory planning policy by utilizing the causal knowledge. Numerical results demonstrate that the performance of the proposed algorithm is superior in terms of successful probability, energy consumption, and data loss.