Optimal couple-group tracking control for the heterogeneous multi-agent systems with cooperative-competitive interactions via reinforcement learning method

In this paper, we study a class of optimal couple-group tracking control (OCGTC) problems for heterogeneous multi-agent systems (HeMASs) based on reinforcement learning (RL) method, whose goal is to minimize the local tracking errors (states) and control inputs (actions) of followers by learning the dynamic knowledge of a single leader. The weakly connected multi-agent network is randomly divided into coupled sub-networks, and each agent in the same sub-network cooperates to accomplish tracking control such that the positions and velocities of all the agents converge to the same value, while the agents from different subgroups compete with each other to dissimilar tracking goals. In particular, in the discussed HeMASs, we consider agents with unknown dynamics of first-order and second-order. To solve the algebraic Riccati equation (ARE), an policy-value-based actor-critic technique is applied. Using the Lyapunov-like theorem, we verify that the local tracking error and the estimated weights of actor-critic neural networks are deduced to be uniformly ultimately bounded. Eventually, several simulations demonstrate the correctness of the retrieved theoretical results.