Developing agile behaviors for legged robots remains a challenging problem. While deep reinforcement learning is a promising approach, learning truly agile behaviors typically requires tedious reward shaping and careful curriculum design. We formulate agile locomotion as a multi-stage learning problem in which a mentor guides the agent throughout the training. The mentor is optimized to place a checkpoint to guide the movement of the robot's center of mass while the student (i.e. the robot) learns to reach these checkpoints. Once the student can solve the task, we teach the student to perform the task without the mentor. We evaluate our proposed learning system with a simulated quadruped robot on a course consisting of randomly generated gaps and hurdles. Our method significantly outperforms a single-stage RL baseline without a mentor, and the quadruped robot can agilely run and jump across gaps and obstacles. Finally, we present a detailed analysis of the learned behaviors' feasibility and efficiency.
ATRos: Learning Energy-Efficient Agile Locomotion for Wheeled-legged Robots
Shanghai Huawei Technologies Co., Ltd.（上海华为技术有限公司）；College of Future Information Technology, Fudan University（复旦大学未来信息科技学院）；Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）；School of Aeronautic and Astronautics, Zhejiang University（浙江大学航空宇航学院）；School of Electronics and Communication Engineering, Sun Yat-sen University（中山大学电子与通信工程学院）
Developing controllers for agile locomotion is a long-standing challenge for legged robots. Reinforcement learning (RL) and Evolution Strategy (ES) hold the promise of automating the design process of such controllers. However, dedicated and careful human effort is required to design training environments to promote agility. In this paper, we present a multi-agent learning system, in which a quadruped robot (protagonist) learns to chase another robot (adversary) while the latter learns to escape. We find that this adversarial training process not only encourages agile behaviors but also effect…
We present an open-source Visual-Inertial-Leg Odometry (VILO) state estimation solution, Cerberus, for legged robots that estimates position precisely on various terrains in real time using a set of standard sensors, including stereo cameras, IMU, joint encoders, and contact sensors. In addition to estimating robot states, we also perform online kinematic parameter calibration and contact outlier rejection to substantially reduce position drift. Hardware experiments in various indoor and outdoor environments validate that calibrating kinematic parameters within the Cerberus can reduce estimati…
Learning and Adapting Agile Locomotion Skills by Transferring Experience
UC Berkeley（加州大学伯克利分校）；Berkeley AI Research（伯克利人工智能研究院）；Google Research（谷歌研究院）；Georgia Institute of Technology（佐治亚理工学院）；Simon Fraser University（西蒙菲莎大学）
University of Trento（特伦托大学）；Scuola Superiore Sant’Anna（圣安娜高等学校）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）
Robots need lightweight legs for agile locomotion, and intrinsic series elastic compliance has proven to be a major ingredient for energy-efficient locomotion and robust locomotion control. Animals' anatomy and locomotion capabilities emphasize the importance of that lightweight legs and integrated, compact, series elastically actuated for distal leg joints. But unlike robots, animals achieve series elastic actuation by their muscle-tendon units. So far no designs are available that feature all characteristics of a perfect distal legged locomotion actuator; a low-weight and low-inertia design,…
Heriot-Watt University（赫瑞-瓦特大学）；University of Edinburgh（爱丁堡大学）；University of Oxford（牛津大学）；Dalian University of Technology（大连理工大学）
Planning and execution of agile locomotion maneuvers have been a longstanding challenge in legged robotics. It requires to derive motion plans and local feedback policies in real-time to handle the nonholonomy of the kinetic momenta. To achieve so, we propose a hybrid predictive controller that considers the robot's actuation limits and full-body dynamics. It combines the feedback policies with tactile information to locally predict future actions. It converges within a few milliseconds thanks to a feasibility-driven approach. Our predictive controller enables ANYmal robots to generate agile m…
Learning Quadruped Locomotion using Bio-Inspired Neural Networks with Intrinsic Rhythmicity
Shenzhen Amigaga Technology Co Ltd.（深圳阿米加加科技有限公司）；Sun Yat-sen University（中山大学）；Shenyang Institute of Automation, Chinese Academy of Sciences（中国科学院沈阳自动化研究所）；University College London（伦敦大学学院）
University of Edinburgh（爱丁堡大学）；University College London（伦敦大学学院）
This work developed a kernel-based residual learning framework for quadrupedal robotic locomotion. Initially, a kernel neural network is trained with data collected from an MPC controller. Alongside a frozen kernel network, a residual controller network is trained via reinforcement learning to acquire generalized locomotion skills and resilience against external perturbations. With this proposed framework, a robust quadrupedal locomotion controller is learned with high sample efficiency and controllability, providing omnidirectional locomotion at continuous velocities. Its versatility and robu…
SoloParkour: Constrained Reinforcement Learning for Visual Locomotion from Privileged Experience
LAAS-CNRS（法国国家科学研究中心自动化与系统分析实验室）；Université de Toulouse（图卢兹大学）；CNRS（法国国家科学研究中心）；New York University（纽约大学）；Artificial and Natural Intelligence Toulouse Institute（图卢兹人工与自然智能研究所）
NASA Jet Propulsion Laboratory, California Institute of Technology（美国国家航空航天局喷气推进实验室，加州理工学院）；University of Goettingen, University Medical Center Goettingen（哥廷根大学，哥廷根大学医学中心）；Institute for Robotics and Intelligent Machines, Georgia Institute of Technology（佐治亚理工学院机器人与智能机器研究所）；Swiss Federal Institute of Technology (ETH Zürich)（苏黎世联邦理工学院）；California Institute of Technology（加州理工学院）
Legged robots can traverse a wide variety of terrains, some of which may be challenging for wheeled robots, such as stairs or highly uneven surfaces. However, quadruped robots face stability challenges on slippery surfaces. This can be resolved by adjusting the robot's locomotion by switching to more conservative and stable locomotion modes, such as crawl mode (where three feet are in contact with the ground always) or amble mode (where one foot touches down at a time) to prevent potential falls. To tackle these challenges, we propose an approach to learn a model from past robot experience for…
Evaluation of an Actuated Spine in Agile Quadruped Locomotion
Department of Computer Science, Technical University of Darmstadt（德意志技术大学达姆施塔特分校计算机科学系）；IDEAS Research Institute（IDEAS研究所）；Institute of Robotics and Machine Intelligence, Poznan University of Technology（波兹南技术大学机器人与机器智能研究所）
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Tandon School of Engineering, New York University（纽约大学坦登工程学院）；Oxford Robotics Institute, University of Oxford（牛津大学牛津机器人研究所）
Implementing dynamic locomotion behaviors on legged robots requires a high-quality state estimation module. Especially when the motion includes flight phases, state-of-the-art approaches fail to produce reliable estimation of the robot posture, in particular base height. In this paper, we propose a novel approach for combining visual-inertial odometry (VIO) with leg odometry in an extended Kalman filter (EKF) based state estimator. The VIO module uses a stereo camera and IMU to yield low-drift 3D position and yaw orientation and drift-free pitch and roll orientation of the robot base link in t…
University of Aveiro（阿威罗大学）；University of Porto（波尔图大学）
Humanoid robots are made to resemble humans but their locomotion abilities are far from ours in terms of agility and versatility. When humans walk on complex terrains, or face external disturbances, they combine a set of strategies, unconsciously and efficiently, to regain stability. This paper tackles the problem of developing a robust omnidirectional walking framework, which is able to generate versatile and agile locomotion on complex terrains. The Linear Inverted Pendulum Model and Central Pattern Generator concepts are used to develop a closed-loop walk engine, which is then combined with…
TAGA: Terrain-aware Active Gaze Learning for Generalizable Agile Humanoid Locomotion
MarmotLab, National University of Singapore（马尔莫实验室，新加坡国立大学）；Center of X-Mechanics, Zhejiang University（浙大X力学中心）；South China University of Technology（华南理工大学）
Max Planck Institute of Intelligent Systems（马克斯·普朗克智能系统研究所）
Bipedal animals have diverse morphologies and advanced locomotion abilities. Terrestrial birds, in particular, display agile, efficient, and robust running motion, in which they exploit the interplay between the body segment masses and moment of inertias. On the other hand, most legged robots are not able to generate such versatile and energy-efficient motion and often disregard trunk movements as a means to enhance their locomotion capabilities. Recent research investigated how trunk motions affect the gait characteristics of humans, but there is a lack of analysis across different bipedal mo…
Collaborative Robotics and Intelligent Systems Institute, Oregon State University（俄勒冈州立大学协作机器人与智能系统研究所）
The complex dynamics of agile robotic legged locomotion requires motion planning to intelligently adjust footstep locations. Often, bipedal footstep and motion planning use mathematically simple models such as the linear inverted pendulum, instead of dynamically-rich models that do not have closed-form solutions. We propose a real-time optimization method to plan for dynamical models that do not have closed form solutions and experience irrecoverable failure. Our method uses a data-driven approximation of the step-to-step dynamics and of a failure margin function. This failure margin function …
Diffusion-based learning of contact plans for agile locomotion
Munich Institute of Robotics and Machine Intelligence, Technical University of Munich, Germany（慕尼黑机器人与机器智能研究所，慕尼黑技术大学）；Tandon School of Engineering, New York University, USA（纽约大学土曼工学院）；Max-Planck Institute for Intelligent Systems, Tübingen, Germany（马克斯·普朗克智能系统研究所，图宾根）
Agile and Generalized Legged Locomotion via Attention-Based Neural Map Encoding
Robotic Systems Lab, ETH Zurich, Switzerland（机器人系统实验室，苏黎世联邦理工学院，瑞士）；Secure, Reliable, and Intelligent Systems Lab, ETH Zurich, Switzerland（安全、可靠和智能系统实验室，苏黎世联邦理工学院，瑞士）；ETH AI Center, Switzerland（苏黎世联邦理工学院人工智能中心，瑞士）
Institute for Robotics and Intelligent Machines, Georgia Institute of Technology（佐治亚理工学院机器人与智能机器研究所）；MIT Biomimetic Robotics Lab（麻省理工学院仿生机器人实验室）；NAVER LABS（NAVER实验室）
In this letter, we present a versatile hierarchical offline planning algorithm, along with an online control pipeline for agile quadrupedal locomotion. Our offline planner alternates between optimizing centroidal dynamics for a reduced-order model and whole-body trajectory optimization, with the aim of achieving dynamics consensus. Our novel momentum-inertia-aware centroidal optimization, which uses an equimomental ellipsoid parameterization, is able to generate highly acrobatic motions via ``inertia shaping". Our whole-body optimization approach significantly improves upon the quality of stan…
Walk the PLANC: Physics-Guided RL for Agile Humanoid Locomotion on Constrained Footholds
Department of Mechanical and Civil Engineering, California Institute of Technology（机械与土木工程系，加州理工学院）；Department of Computing and Mathematical Sciences, California Institute of Technology（计算与数学科学系，加州理工学院）
Unleashing the Agility of Wheeled-Legged Robots for High-Dynamic Reflexive Obstacle Evasion
School of Mechanical Engineering, Tianjin University（天津大学机械工程学院）；School of Computing, National University of Singapore（新加坡国立大学计算机学院）；Beijing Zhongguancun Academy（北京中关村学院）；DeepRobotics（深睿科技）
Legged robots pose one of the greatest challenges in robotics. Dynamic and agile maneuvers of animals cannot be imitated by existing methods that are crafted by humans. A compelling alternative is reinforcement learning, which requires minimal craftsmanship and promotes the natural evolution of a control policy. However, so far, reinforcement learning research for legged robots is mainly limited to simulation, and only few and comparably simple examples have been deployed on real systems. The primary reason is that training with real robots, particularly with dynamically balancing systems, is …
Google Research（谷歌研究院）；University of California, Berkeley（加利福尼亚大学伯克利分校）
Reproducing the diverse and agile locomotion skills of animals has been a longstanding challenge in robotics. While manually-designed controllers have been able to emulate many complex behaviors, building such controllers involves a time-consuming and difficult development process, often requiring substantial expertise of the nuances of each skill. Reinforcement learning provides an appealing alternative for automating the manual effort involved in the development of controllers. However, designing learning objectives that elicit the desired behaviors from an agent can also require a great dea…
Robots operating in human environments need various skills, like slow and fast walking, turning, side-stepping, and many more. However, building robot controllers that can exhibit such a large range of behaviors is a challenging problem that requires tedious investigation for every task. We present a unified model-based control algorithm for imitating different animal gaits without expensive simulation training or real-world fine-tuning. Our method consists of stance and swing leg controllers using a centroidal dynamics model augmented with online adaptation techniques. We also develop a whole…
KYON: Semi-Modular Wheel-Legged Quadruped With Agile Bimanual Capability
Humanoids and Human Centered Mechatronics, Fondazione Istituto Italiano di Tecnologia（人机一体化机械系统，意大利理工研究院）；Advanced Robotics Facility, Fondazione Istituto Italiano di Tecnologia（先进机器人设施，意大利理工研究院）
BeamDojo: Learning Agile Humanoid Locomotion on Sparse Footholds
Shanghai AI Laboratory（上海人工智能实验室）；Shanghai Jiao Tong University（上海交通大学）；Zhejiang University（浙江大学）；The University of Hong Kong（香港大学）；The Chinese University of Hong Kong（香港中文大学）
National University of Defense Technology（国防科技大学）；College of Intelligence Science and Technology（智能科学与技术学院）；Robotics Research Center（机器人研究中心）
In this paper, we propose a robust controller that achieves natural and stably fast locomotion on a real blind quadruped robot. With only proprioceptive information, the quadruped robot can move at a maximum speed of 10 times its body length, and has the ability to pass through various complex terrains. The controller is trained in the simulation environment by model-free reinforcement learning. In this paper, the proposed loose neighborhood control architecture not only guarantees the learning rate, but also obtains an action network that is easy to transfer to a real quadruped robot. Our res…
