Optimized Formation Control for Multi-Agent Systems Based on Adaptive Dynamic Programming Without Persistence of Excitation

In this letter, an adaptive dynamic programming (ADP) method is proposed for optimized formation control of second-order linear systems. The method exploits an actor-critic architecture, where an actor component is used to learn the optimal formation controller, and a critic component is used to learn the optimal value function. Generally, ADP requires a priori knowledge of persistence of excitation (PE) to guarantee the stability of the control system. However, the PE condition is hard to verify during the learning process and in practical applications. To this end, this letter redesigns the updating laws of the actor and critic components to ensure that the Bellman residual error can eventually approach to zero, and the stability of the control system can be guaranteed without introducing the PE and additional constraints. By using Lyapunov stability analysis, we prove that the proposed optimized formation scheme can achieve the desired optimizing performance. Finally, a simulation example is given to demonstrate the effectiveness of the proposed method.