Journal of Bionic Engineering (2024) 21:1278–1289https://doi.org/10.1007/s42235-023-00452-9
Learning Robust Locomotion for Bipedal Robot via Embedded Mechanics Properties
Yuanxi Zhang1 · Xuechao Chen1,2 · Fei Meng1,2 · Zhangguo Yu1,2 · Yidong Du1 · Junyao Gao1,2 · Qiang Huang1,2
1 School of Mechatronical Engineering, Beijing Institute of Technology, Beijing 100081, China
2 Key Laboratory of Biomimetic Robots and Systems, Ministry of Education, Beijing 100081, China
Abstract
Reinforcement learning (RL) provides much potential for locomotion of legged robot. Due to the gap between simulation and the real world, achieving sim-to-real for legged robots is challenging. However, the support polygon of legged robots can help to overcome some of these challenges. Quadruped robot has a considerable support polygon, followed by bipedal robot with actuated feet, and point-footed bipedal robot has the smallest support polygon. Therefore, despite the existing sim-to-real gap, most of the recent RL approaches are deployed to the real quadruped robots that are inherently more stable, while the RL-based locomotion of bipedal robot is challenged by zero-shot sim-to-real task. Especially for the point-footed one that gets better dynamic performance, the inevitable tumble brings extra barriers to sim-to-real task. Actually, the crux of this type of problem is the diference of mechanics properties between the physical robot and the simulated one, making it difcult to play the learned skills well on the physical bipedal robot. In this paper, we introduce the embedded mechanics properties (EMP) based on the optimization with Gaussian processes to RL training, making it possible to perform sim-toreal transfer on the BRS1-P robot used in this work, hence the trained policy can be deployed on the BRS1-P without any struggle. We validate the performance of the learning-based BRS1-P on the condition of disturbances and terrains not ever learned, demonstrating the bipedal locomotion and resistant performance.
Keywords Bipedal robot · Reinforcement learning · Sim-to-real · Mechanics properties