In this work, we propose a learning approach for 3D dynamic bipedal walking when footsteps are constrained to stepping stones. While recent work has shown progress on this problem, real-world demonstrations have been limited to relatively simple open-loop, perception-free scenarios. Our main contribution is a more advanced learning approach that enables real-world demonstrations, using the Cassie robot, of closed-loop dynamic walking over moderately difficult stepping-stone patterns. Our approach first uses reinforcement learning (RL) in simulation to train a controller that maps footstep commands onto joint actions without any reference motion information. We then learn a model of that controller's capabilities, which enables prediction of feasible footsteps given the robot's current dynamic state. The resulting controller and model are then integrated with a real-time overhead camera system for detecting stepping stone locations. For evaluation, we develop a benchmark set of stepping stone patterns, which are used to test performance in both simulation and the real world. Overall, we demonstrate that sim-to-real learning is extremely promising for enabling dynamic locomotion over stepping stones. We also identify challenges remaining that motivate important future research directions.
Robust Humanoid Walking on Compliant and Uneven Terrain with Deep Reinforcement Learning
CNRS-AIST JRL (Joint Robotics Laboratory) IRL, National Institute of Advanced Industrial Science and Technology (AIST), Japan（日本国家先进工业科学与技术研究院（AIST））；University of Tsukuba（筑波大学）；Department of Computer Science, Stanford University（斯坦福大学计算机科学系）
Adaptive Step Duration for Accurate Foot Placement: Achieving Robust Bipedal Locomotion on Terrains with Restricted Footholds
Mechanical and Aerospace Engineering, The Ohio State University（机械与航空航天工程系，俄亥俄州立大学）；Electrical and Computer Engineering, The Ohio State University（电气与计算机工程系，俄亥俄州立大学）
Nonlinear Model Predictive Control for Robust Bipedal Locomotion: Exploring Angular Momentum and CoM Height Changes
School of Power and Mechanical Engineering, Wuhan University（武汉大学动力与机械学院）；Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳人工智能与机器人研究院）；School of Mechanical Engineering, University of Leeds（利兹大学机械工程学院）；School of Informatics, The University of Edinburgh（爱丁堡大学信息学院）；Humanoid and Human Centered Mechatronics Research Line, Istituto Italiano di Tecnologia（意大利技术研究院人形与以人为中心的机电一体化研究线）
Learning Linear Policies for Robust Bipedal Locomotion on Terrains with Varying Slopes
Indian Institute of Technology (BHU) Varanasi（印度理工学院（巴纳拉斯印度教大学）瓦拉纳西分校）；Indian Institute of Technology Roorkee（印度理工学院鲁尔基分校）；Ohio State University（俄亥俄州立大学）；Indian Institute of Science（印度科学学院）
Bipedal Robust Walking on Uneven Footholds: Piecewise Slope LIPM with Discrete Model Predictive Control
Faculty of Computing, Harbin Institute of Technology（哈尔滨工业大学计算学部）；State Key Laboratory of Robotics and System, Harbin Institute of Technology（哈尔滨工业大学机器人技术与系统国家重点实验室）
Walk the PLANC: Physics-Guided RL for Agile Humanoid Locomotion on Constrained Footholds
Department of Mechanical and Civil Engineering, California Institute of Technology（机械与土木工程系，加州理工学院）；Department of Computing and Mathematical Sciences, California Institute of Technology（计算与数学科学系，加州理工学院）
Enhancing Model-Based Step Adaptation for Push Recovery through Reinforcement Learning of Step Timing and Region
Automation and Control Institute, TU Wien（维也纳工业大学自动化与控制研究所）；Autonomous Systems Lab, TU Wien（维也纳工业大学自主系统实验室）；Institute of Robotics and Mechatronics (DLR), German Aerospace Center（德国航空航天中心机器人与机电一体化研究所（DLR））
