Iterative Q-Learning for Model-Free Optimal Control With Adjustable Convergence Rate

In this paper, a novel accelerated Q-learning algorithm is developed to address optimal control problems for discrete-time nonlinear systems. First, the accelerated Q-learning scheme is proposed by introducing the relaxation factor. Note that the relaxation factor leads to the adjustability of the convergence rate. Second, the convergence of the Q-function is analyzed with different relaxation factors. Third, the adjustable Q-learning scheme is developed with guaranteed convergence, which can adaptively change the value of the relaxation factor. Finally, the simulation results demonstrate the effectiveness of this proposed algorithm.