Sliding mode heading control for AUV based on continuous hybrid model-free and model-based reinforcement learning

For autonomous underwater vehicles (AUVs), control over AUV heading is of key importance to enable high-performance locomotion control. In this study, the heading control is achieved by using the robust sliding mode control (SMC) method. The performance of the controller can be seriously affected by its parameters. However, it is time-consuming and labor-intensive to manually adjust the parameters. Most of the existing methods rely on the accurate AUV model or prior knowledge, which are difficult to obtain. Therefore, this study is concerned with the problem of automatically tuning the SMC parameters through reinforcement learning (RL). First, an AUV dynamic model with and without current influence was successfully established. Second, a continuous hybrid Model-based Model-free (MbMf) RL method based on the deterministic policy gradient was introduced and explained. Then, the framework for tuning the parameters of SMC by the RL method was described. Finally, to demonstrate the robustness and effectiveness of our approach, extensive numerical simulations were conducted on the established AUV model. The results show that our method can automatically tune the SMC parameters. The performance is more effective than SMC with fixed parameters or SMC with a purely model-free learner.