Blind Walking Balance Control and Disturbance Rejection of the Bipedal Humanoid Robot Xiao-Man via Reinforcement Learning

Bipedal humanoid robot has the ability to both move and manipulate in complex environments, which is of great significance in the future. However, stable bipedal walking in the real world has always been a challenge in industry and even in academia. The traditional model-based methods are highly dependent on the environment, with high modeling complexity and lack of generalization. The solution based on the simplified model usually causes the problem that the control algorithms cannot adapt to complex terrain environment. This paper presents a newly designed bipedal humanoid robot, Xiao-Man. Aiming at achieving the robot's terrain-adaptive walking behavior, a reinforcement learning based Actor-Critic network with asymmetric structure is proposed. Without using any external perception information, robust bipedal walking behavior of Xiao-Man is achieved. In the process, we also build the dataset based on the joint actuation truth data and train a joint actuator network to reduce the gap between the expected torque and the actual response torque. Experimental results show that the bipedal humanoid robot equipped with the trained control policy achieves the capability of stable walking and disturbance rejection only rely on proprioceptive information.