Reinforcement Learning Formation Tracking of Networked Autonomous Surface Vehicles With Bounded Inputs via Cloud-Supported Communication

This paper investigates formation tracking (FT) problem of the networked autonomous surface vehicles (NASVs) with bounded inputs. In order to achieve distributed control, a prescribed-time observer is employed to reshape the leader's states for the follower ASVs, which can only receive the message from the neighbor ASVs. For reducing communication costs and the negative effect of bounded inputs and the unknown uncertainties, a hierarchical reinforcement learning control (HRLC) algorithm based on the cloud-supported communication is proposed, where the cloud-supported estimator is constructed such that the estimated states approach the leader's states with the less communication costs. The local reinforcement learning controller is designed according to the actor-critic strategy such that the actual states converge to the estimated states with the given formation offset. With the help of Lyapunov stability and Hurwitz stability theory, some sufficient conditions of the close-loop system have be obtained. Finally, simulation examples have be proposed to validate the theoretical analysis.