Tracking moving target for 6 degree-of-freedom robot manipulator with adaptive visual servoing based on deep reinforcement learning PID controller

In this paper, an image-based visual servoing (IBVS) controller with a 6 degree-of-freedom robotic manipulator that tracks moving objects is investigated using the proposed Deep Q-Networks and proportional-integral-derivative (DQN-PID) controller. First, the classical IBVS controller and the problem of feature loss and large steady-state error for tracking moving targets are introduced. Then, a DQN-PID based IBVS method is proposed to solve the problem of feature loss and large steady-state error and improve the servo precision, as the existing methods are hard to use for solve the problems. Specifically, the IBVS method is inherited by our controller to build the tracking model, and a value-based reinforcement learning method is proposed as an adaptive law for dynamically tuning the PID parameters in the discrete space, which can track the moving target and keep the servo feature in the field of the camera. Finally, compared with the different existing methods, the DQN-PID based IBVS method has merits of higher accuracy and more stable tracking, or generalization.