Dynamic Trajectory Planning of a 7-DOF Surgical Robot Based on HER-DDPG Algorithm

Abstract Traditional trajectory planning approaches are currently lacking in intelligence and autonomy. We used the reinforcement learning approach to solve the autonomous trajectory planning of the robot arm to avoid obstacles with uniform motion and hit the target point quickly with obstacle avoidance planning for surgical robots taken as the practical background. We used the algorithm of experience playback mechanism combined with off-policy DDPG based on reinforcement learning, and after several iterations, the robot completed trajectory planning with obstacle avoidance autonomously. Moving obstacles were added to roughly simulate the autonomous obstacle avoidance of a surgical robotic arm with moving medical personnel or mobile instruments in the operating room, based on the simple trajectory planning example of Open-AI Open-Source Project Baseline, combined with the research context. Sparse rewards were used for each iteration based on the HER algorithm, so that each attempt could gain experience. The HER-DDPG method can quickly complete the manipulator’s trajectory planning in a simulation environment, which is critical for the surgical robot’s autonomous positioning in the real world. Furthermore, the experience playback system has been tested to allow full use of sparse rewards and handle parallel tasks equally well.