Adaptive Fault-Tolerant Tracking Control for Affine Nonlinear Systems With Unknown Dynamics via Reinforcement Learning

This paper investigates the optimal fault-tolerant tracking control (FTTC) problem for unknown affine nonlinear continuous-time systems with process and actuator faults in the framework of reinforcement learning (RL). The proposed novel active FTTC scheme is based on adaptive optimal control theory. In this way, the FTTC problem is formulated as an optimal regulation problem for the augmented system, which consists of the controlled system and the reference trajectory. To solve the Hamilton-Jacobi-Bellman (HJB) equation of the augmented system, an identifier-critic-based online RL strategy is employed with a dual neural network (NN) approximation structure. Initially, in order to remove the requirement of prior knowledge of the system dynamics, an adaptive NN identifier is designed. The forgetting factor in the proposed identifier update law is variable and a function of the filtered state estimation error and filtered state error. Choosing this variable forgetting factor increases the convergence speed and decreases the estimation error of identifier NN weights compared to the constant one while maintaining its robustness. When a fault occurs, the system continues to operate under the former FTTC until the fault is detected. Meanwhile, the optimal FTTC design in the RL framework requires the initial admissible control condition. In order to make it possible to initiate the FTTC learning process from the former FTTC, we employed a stabilizing term in the critical update rule. The Uniformly Ultimately Boundedness (UUB) of identifier and critic NN weight errors and, as a result, the convergence of the control input to the neighborhood of the optimal solution are all proved by Lyapunov theory. In the proposed method, changes in the values of faults are detected by comparing the HJB error to a predefined threshold. Finally, the simulation results are given to validate the effectiveness of the developed method. Note to Practitioners—Long-time operations and the influence of external perturbations often make the faults inevitable for many practical engineering systems which can lead to unpredictable behaviors and catastrophic impacts. In general, the faults are naturally uncertain in time, value, and pattern, that is, it is unknown when, how much, and which system components fail. Therefore, the control system must be able to tolerate an extensive set of component faults. The design of optimal model-free FTTC strategies in an adaptive manner is challenging in nonlinear systems. The proposed method is suitable for a large class of nonlinear systems with input-affine form, and guarantees the system stability in the presence of process and actuator faults.