Distributed safe formation maneuver control of Euler–Lagrange multi-agent systems in a partially unknown environment by safe reinforcement learning

This paper describes a multi-layer approach to the problem of safe formation control. The agents’ and the leader’s dynamics are considered unknown Euler–Lagrange (E-L) systems. In addition, the environment is partially unknown. We propose a novel layered approach to reach the predefined target while preserving a designed, safe, optimal formation pattern along a planned optimal path. By satisfying the safety constraints, safe reinforcement learning (RL) is introduced to ensure the leader reaches the desired destination without collision. Maintaining a constant formation pattern is unsafe for followers since they are not familiar with the surroundings. Thus, we define the formation maneuver control problem, which can adjust formation geomatical patterns dynamically depending on the environment. A proposed algorithm based on the leader’s designed path is defined to solve the problem. Using off-policy RL, the model-free distributed control law is presented to generate a designed formation pattern in a determined optimal path. Finally, we demonstrate that the proposed approach can be applied to the safe formation maneuver problem in an environment with convex obstacles. This paper presents a safe formation control strategy that addresses practical issues, such as model uncertainty, without requiring sensor measurements in an unknown, static environment without uncertainty. Simulation demonstrates the effectiveness of the suggested approaches for a group of Uncrewed Surface Vehicles (USVs).