Optimized Formation Control for a Class of Second-order Multi-agent Systems based on Single Critic Reinforcement Learning Method

In this paper, an optimized formation control based on single critic reinforcement learning is developed for a class of second-order multi-agent systems. Unlike first-order systems, both position and velocity variables need to be considered in second-order system control. Therefore, the control of second-order systems is more challenging. In the control design, single critic reinforcement learning method combined with fuzzy logic systems is used. Fuzzy logic systems approximator is used to compensate the nonlinearity of the systems. Compared with the actor-critic reinforcement learning method, single critic reinforcement learning requires only one network iterative training such that the training errors are smaller, and the calculation time caused by the iterative loop between actor and critic can be reduced. According to the analysis of Lyapunov stability theory, the proposed control design can achieve the control objective. Finally, the effectiveness of the proposed method is verified by simulation.