Optimal Control of Unmanned Surface Vehicle Trajectory Tracking Based on Reinforcement Learning

For the purpose of achieving high dynamic and precise tracking of desired trajectory, an optimal trajectory tracking control strategy with specified performance based on reinforcement learning was proposed for the unmanned surface vehicle (USV) trajectory tracking system. For the MIMO discrete-time system of USV, in order to constrain its tracking dynamic error within the expected specified range to ensure high dynamic performance during the trajectory tracking process, the system performance index and the long-term cost function were designed to measure the performance. On this basis, a control system based on the Reinforcement Learning Actor-critic framework is constructed, in which two neural networks are applied. The actor NN is used to generate the optimal control signal, and the critic NN is used to evaluate the performance of the USV while approximating the cost function and measuring actor NN. The weight of the two NNs is directly adjusted during the operation of the USV. A strict theoretical analysis is given for the designed control system, and it is proved that the closed-loop system is stable, and all closed-loop signals are semi-globally consistent and ultimately bounded. Finally, it is verified by simulation that the control system can well realize the trajectory tracking of the USV model.