Footstep reward for energy-efficient quadruped gait generation and transition through deep reinforcement learning

The generation of quadruped robot gait through deep reinforcement learning (DRL) has emerged as a prominent topic in recent years. However, DRL methods face challenges in generating specific and energy-efficient gait patterns, as well as in achieving gait transition. These methods often require imposing restrictions on movement or on the learning process to achieve proper gait specification and generation, thereby limiting the quadruped model movement achievable with DRL training. In this paper, we propose a new DRL reward design method, called FootStep Reward, to successfully generate specific gait patterns that exhibit animal-like energetic properties. Our method is capable of generating Walk, Trot, and Gallop gait patterns at any velocity while demonstrating better energy efficiency compared to similar gait specification methods. Additionally, we employ our method to analyze the impact of two body parameters: model mass and joint stiffness, increasing our understanding of how these parameters influence Walk, Trot and Gallop gait efficiency. Finally, we implement the FootStep Reward method to generate gait transitions and successfully achieve the transition from Walk to Trot gait, optimizing locomotion at low and medium velocities.