Optimal robust formation control for heterogeneous multi‐agent systems based on reinforcement learning

Abstract In this article, a reinforcement learning (RL)‐based robust control strategy is proposed for uncertain heterogeneous multi‐agent systems to achieve optimal collision‐free time‐varying formations. Without using any global information, a fully distributed adaptive observer is developed to estimate both dynamics and states of the reference and disturbance systems. The observer parameters are found by an observed model‐based or a model‐free off‐policy RL algorithm. Using the internal model principle, a novel optimal robust formation control strategy is developed based on another proposed off‐policy RL algorithm. The algorithm addresses the nonquadratic optimization problem when the system model is completely unknown. Taking the bushfire edge tracking and patrolling task for an unmanned aerial vehicle‐unmanned ground vehicle heterogeneous system as an example, the effectiveness and robustness of the developed control strategy are verified by simulations.