A Novel Adaptive Control Design for a Class of Nonstrict-Feedback Discrete-Time Systems via Reinforcement Learning

In this article, an adaptive reinforcement learning (RL) control problem is explored for a class of nonstrict-feedback discrete-time systems. First, different from the existing results, considering the noncausal problem which may exist in the backstepping design procedure, a universal system transformation method is first proposed for a class of nonstrict-feedback discrete-time systems. Second, by defining a compensation term to compensate the controller and utilizing the property of radial-basis-function neural network (RBFNN), an RL-based direct adaptive control strategy is developed via a backstepping method to achieve optimal control, and the multigradient recursive (MGR) algorithm is employed to estimate the weight vector. Finally, the stability of the control system is guaranteed and all signals in the closed-loop system are semiglobal uniformly ultimately bounded (SGUUB) on the basis of the Lyapunov theory. In addition, a universal system transformation is first proposed which breaks through the limitations on the controller design for the discrete-time nonstrict-feedback nonlinear system by using the traditional method. The validity of this strategy is verified by two simulation examples that include a course keeping system of the marine vessel.