Distributed Optimal Control of Nonlinear System Based on Policy Gradient With External Disturbance

Aiming at the distributed optimal control problem in presence of external disturbance, a data-driven iterative algorithm based on policy gradient is proposed for multi-agent systems (MAS) in this article. Taking into full consideration the influence of itself and local neighbor perturbation information, the optimization problem is replaced by a zero-sum game problem with control and disturbance policies. Then, the control behavior under distributed conditions is evaluated using the action-state value function to achieve decoupling of system parameters. Afterwards, adaptive dynamic programming (ADP) method is designed based on the actor-critic neural network structure to obtain the Nash equilibrium solution of the zero-sum game. The rules for updating the network weights are given by utilizing the residual weighting method based on both online and offline data. Besides, asynchronous update iteration control technique is developed to address the situation that other agents do not update their policies at the same time. Sufficient conditions for stability and convergence are established through a rigorous proof. The validity of the established data-driven control method is demonstrated by the simulation results.