Model-free optimal containment control of multi-agent systems based on actor-critic framework

This paper deals with the model-free optimal containment control problem for a class of linear multi-agent systems (MASs). In the existing results concerning containment control of MASs, the dynamics of the MASs is required to be completely known. Differently, in this paper, we propose a new distributed self-learning control scheme based on action dependent heuristic dynamic programming (ADHDP) to achieve containment control, where the model of MASs is no longer needed. The containment control problem is first transformed into a regulation problem on the dynamics of the designed local containment error. The policy iteration method based on the designed state-action value function (also called the Q-function) is introduced to deal with such a regulation problem. The convergence analysis of this policy iteration method is also given. Neural network (NN) based actor-critic framework is adopted to approximate the optimal Q-functions and the optimal control policies for facilitating the implementation of the proposed method. It shows that the approximated control policies achieve the containment control and satisfy the global Nash equilibrium. Finally, the simulation results are provided to demonstrate the effectiveness of the developed approach.