Adaptive reinforcement learning fault-tolerant control for AUVs With thruster faults based on the integral extended state observer

In this paper, a reinforcement learning (RL) fault-tolerant control (FTC) method is proposed for trajectory tracking of autonomous underwater vehicles (AUVs) with thruster faults. To deal with the thruster fault, unknown disturbance and model uncertainty, a new integral extended state observer (IESO) for fault diagnosis observation is proposed, which uses a conventional ESO to estimate the total system uncertainty, and introduces an integral mechanism to mitigate the effect of estimation error further. Thus, the problem that the estimation error caused by the traditional ESO leads to the decline of the fault-tolerant capability of the FTC system is solved. Then, to solve the problem of integral saturation due to the introduction of the integral term, the integral term is limited after the thruster fault of the AUV. Furthermore, based on the actor–critic structure of RL, a PD-like feedback controller is designed to realize the FTC of AUV in the face of thruster fault by using the total uncertainty of the IESO scheme. And the input saturation of the thruster is considered, and an auxiliary variable system is used to handle the control truncation between saturated and unsaturated inputs. Based on the Lyapunov method, the stability of the closed-loop system is analyzed and proved. Finally, the proposed method is verified to have good fault tolerance and robustness by simulation and underwater experiments.