Legged robots pose one of the greatest challenges in robotics. Dynamic and agile maneuvers of animals cannot be imitated by existing methods that are crafted by humans. A compelling alternative is reinforcement learning, which requires minimal craftsmanship and promotes the natural evolution of a control policy. However, so far, reinforcement learning research for legged robots is mainly limited to simulation, and only few and comparably simple examples have been deployed on real systems. The primary reason is that training with real robots, particularly with dynamically balancing systems, is complicated and expensive. In the present work, we introduce a method for training a neural network policy in simulation and transferring it to a state-of-the-art legged system, thereby leveraging fast, automated, and cost-effective data generation schemes. The approach is applied to the ANYmal robot, a sophisticated medium-dog-sized quadrupedal system. Using policies trained in simulation, the quadrupedal machine achieves locomotion skills that go beyond what had been achieved with prior methods: ANYmal is capable of precisely and energy-efficiently following high-level body velocity commands, running faster than before, and recovering from falling even in complex configurations.
Istituto Italiano di Tecnologia (IIT)（意大利技术研究院）；PAL Robotics（PAL机器人公司）
For legged robots to perform agile, highly dynamic and contact-rich motions, whole-body trajectories computation of under-actuated complex systems subject to non-linear dynamics is required. In this work, we present hands-on applications of Horizon, a novel open-source framework for trajectory optimization tailored to robotic systems, that provides a collection of tools to simplify dynamic motion generation. Horizon was tested on a broad range of behaviours involving several robotic platforms: we introduce its building blocks and describe the complete procedure to generate three complex motion…
Heriot-Watt University（赫瑞-瓦特大学）；University of Edinburgh（爱丁堡大学）；University of Oxford（牛津大学）；Dalian University of Technology（大连理工大学）
Planning and execution of agile locomotion maneuvers have been a longstanding challenge in legged robotics. It requires to derive motion plans and local feedback policies in real-time to handle the nonholonomy of the kinetic momenta. To achieve so, we propose a hybrid predictive controller that considers the robot's actuation limits and full-body dynamics. It combines the feedback policies with tactile information to locally predict future actions. It converges within a few milliseconds thanks to a feasibility-driven approach. Our predictive controller enables ANYmal robots to generate agile m…
ATRos: Learning Energy-Efficient Agile Locomotion for Wheeled-legged Robots
Shanghai Huawei Technologies Co., Ltd.（上海华为技术有限公司）；College of Future Information Technology, Fudan University（复旦大学未来信息科技学院）；Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）；School of Aeronautic and Astronautics, Zhejiang University（浙江大学航空宇航学院）；School of Electronics and Communication Engineering, Sun Yat-sen University（中山大学电子与通信工程学院）
Successful real-world deployment of legged robots would require them to adapt in real-time to unseen scenarios like changing terrains, changing payloads, wear and tear. This paper presents Rapid Motor Adaptation (RMA) algorithm to solve this problem of real-time online adaptation in quadruped robots. RMA consists of two components: a base policy and an adaptation module. The combination of these components enables the robot to adapt to novel situations in fractions of a second. RMA is trained completely in simulation without using any domain knowledge like reference trajectories or predefined …
Google Research（谷歌研究院）；University of California, Berkeley（加利福尼亚大学伯克利分校）
Reproducing the diverse and agile locomotion skills of animals has been a longstanding challenge in robotics. While manually-designed controllers have been able to emulate many complex behaviors, building such controllers involves a time-consuming and difficult development process, often requiring substantial expertise of the nuances of each skill. Reinforcement learning provides an appealing alternative for automating the manual effort involved in the development of controllers. However, designing learning objectives that elicit the desired behaviors from an agent can also require a great dea…
MUJICA: Multi-skill Unified Joint Integration of Control Architecture for Wheeled-Legged Robots
College of Intelligent Robotics and Advanced Manufacturing, Fudan University（智能机器人与先进制造学院，复旦大学）；Power China Huadong Engineering Corporation Limited（中国电力工程顾问集团华东分公司）
Mobile robot navigation in complex and dynamic environments is a challenging but important problem. Reinforcement learning approaches fail to solve these tasks efficiently due to reward sparsities, temporal complexities and high-dimensionality of sensorimotor spaces which are inherent in such problems. We present a novel approach to train action policies to acquire navigation skills for wheel-legged robots using deep reinforcement learning. The policy maps height-map image observations to motor commands to navigate to a target position while avoiding obstacles. We propose to acquire the multif…
University of Trento（特伦托大学）；Scuola Superiore Sant’Anna（圣安娜高等学校）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）
Robots need lightweight legs for agile locomotion, and intrinsic series elastic compliance has proven to be a major ingredient for energy-efficient locomotion and robust locomotion control. Animals' anatomy and locomotion capabilities emphasize the importance of that lightweight legs and integrated, compact, series elastically actuated for distal leg joints. But unlike robots, animals achieve series elastic actuation by their muscle-tendon units. So far no designs are available that feature all characteristics of a perfect distal legged locomotion actuator; a low-weight and low-inertia design,…
Legged locomotion is commonly studied and expressed as a discrete set of gait patterns, like walk, trot, gallop, which are usually treated as given and pre-programmed in legged robots for efficient locomotion at different speeds. However, fixing a set of pre-programmed gaits limits the generality of locomotion. Recent animal motor studies show that these conventional gaits are only prevalent in ideal flat terrain conditions while real-world locomotion is unstructured and more like bouts of intermittent steps. What principles could lead to both structured and unstructured patterns across mammal…
Learning and Adapting Agile Locomotion Skills by Transferring Experience
UC Berkeley（加州大学伯克利分校）；Berkeley AI Research（伯克利人工智能研究院）；Google Research（谷歌研究院）；Georgia Institute of Technology（佐治亚理工学院）；Simon Fraser University（西蒙菲莎大学）
Robotics at Google（谷歌机器人）；Georgia Institute of Technology（佐治亚理工学院）
Developing agile behaviors for legged robots remains a challenging problem. While deep reinforcement learning is a promising approach, learning truly agile behaviors typically requires tedious reward shaping and careful curriculum design. We formulate agile locomotion as a multi-stage learning problem in which a mentor guides the agent throughout the training. The mentor is optimized to place a checkpoint to guide the movement of the robot's center of mass while the student (i.e. the robot) learns to reach these checkpoints. Once the student can solve the task, we teach the student to perform …
Saratoga High School（萨拉托加高中）；University of Michigan（密歇根大学）
We propose a mechanically simple and cheap design for a series elastic actuator with controllable stiffness. Such characteristics are necessary for animals for running, jumping, throwing, and manipulation, yet in robots, variable stiffness actuators are either complicated or mimicked at low bandwidth through feedback controllers. A robust and simple design is needed to build reliable and cheap robots with animal capabilities. The key insight of our design is attaching torsional springs to timing belts to create a variable stiffness linear spring. In an antagonistic pair, varying the distance b…
NASA Jet Propulsion Laboratory, California Institute of Technology（美国国家航空航天局喷气推进实验室，加州理工学院）；University of Goettingen, University Medical Center Goettingen（哥廷根大学，哥廷根大学医学中心）；Institute for Robotics and Intelligent Machines, Georgia Institute of Technology（佐治亚理工学院机器人与智能机器研究所）；Swiss Federal Institute of Technology (ETH Zürich)（苏黎世联邦理工学院）；California Institute of Technology（加州理工学院）
Legged robots can traverse a wide variety of terrains, some of which may be challenging for wheeled robots, such as stairs or highly uneven surfaces. However, quadruped robots face stability challenges on slippery surfaces. This can be resolved by adjusting the robot's locomotion by switching to more conservative and stable locomotion modes, such as crawl mode (where three feet are in contact with the ground always) or amble mode (where one foot touches down at a time) to prevent potential falls. To tackle these challenges, we propose an approach to learn a model from past robot experience for…
Developing controllers for agile locomotion is a long-standing challenge for legged robots. Reinforcement learning (RL) and Evolution Strategy (ES) hold the promise of automating the design process of such controllers. However, dedicated and careful human effort is required to design training environments to promote agility. In this paper, we present a multi-agent learning system, in which a quadruped robot (protagonist) learns to chase another robot (adversary) while the latter learns to escape. We find that this adversarial training process not only encourages agile behaviors but also effect…
Unleashing the Agility of Wheeled-Legged Robots for High-Dynamic Reflexive Obstacle Evasion
School of Mechanical Engineering, Tianjin University（天津大学机械工程学院）；School of Computing, National University of Singapore（新加坡国立大学计算机学院）；Beijing Zhongguancun Academy（北京中关村学院）；DeepRobotics（深睿科技）
Space Control and Inertial Technology Research Center（空间控制与惯性技术研究中心）；Harbin Institute of Technology（哈尔滨工业大学）
Impact mitigation is crucial to the stable locomotion of legged robots, especially in high-speed dynamic locomotion. This paper presents a leg locomotion system including the nonlinear active compliance control and the active impedance control for the steel wire transmission-based legged robot. The developed control system enables high-speed dynamic locomotion with excellent impact mitigation and leg position tracking performance, where three strategies are applied. a) The feed-forward controller is designed according to the linear motor-leg model with the information of Coulomb friction and v…
Faculty of Mechanical Engineering, University of Tabriz（大不里士大学机械工程学院）；School of Engineering Emerging Technologies（新兴技术工程学院）
Ants are six-legged insects that can carry loads ten times heavier than their body weight. Since having six-legs, they are intrinsically stable. They are powerful and can carry heavy loads. For these reasons, in this paper a new parallel kinematic structure is proposed for a six-legged ant robot. The mechanical structure is designed and optimized in Solidworks. The mechanism has six legs and only two DC motors actuate the six legs so from mechanical point of view the design is an optimal one. The robot is lightweight and semiautonomous due to using wireless modules. This feature makes this rob…
The functional demands of robotic systems often require completing various tasks or behaviors under the effect of disturbances or uncertain environments. Of increasing interest is the autonomy for dynamic robots, such as multirotors, motor vehicles, and legged platforms. Here, disturbances and environmental conditions can have significant impact on the successful performance of the individual dynamic behaviors, referred to as "motion primitives". Despite this, robustness can be achieved by switching to and transitioning through suitable motion primitives. This paper contributes such a method b…
PIE: Parkour with Implicit-Explicit Learning Framework for Legged Robots
Institute of Cyber-Systems and Control, Zhejiang University（浙江大学控制科学与工程学院 cyber-system 研究所）；State Key Laboratory of Industrial Control Technology（工业控制技术国家重点实验室）
Whole-Body Dynamic Throwing with Legged Manipulators
School of Electrical Engineering and Computer Science, University of Queensland（昆士兰大学电气工程与计算机科学学院）；CSIRO Robotics, DATA61, CSIRO（澳大利亚联邦科学与工业研究组织机器人部 DATA61）
Comparison between External and Internal Single Stage Planetary gearbox actuators for legged robots
Indian Institute of Science（印度科学学院）；Indian Institute of Technology Roorkee（印度理工学院罗尔基分校）；National Institute of Technology Tiruchirappalli（Tiruchirappalli国家理工学院）
Adaptive control can address model uncertainty in control systems. However, it is preliminarily designed for tracking control. Recent advancements in the control of quadruped robots show that force control can effectively realize agile and robust locomotion. In this paper, we present a novel adaptive force-based control framework for legged robots. We introduce a new architecture in our proposed approach to incorporate adaptive control into quadratic programming (QP) force control. Since our approach is based on force control, it also retains the advantages of the baseline framework, such as r…
An attached arm can significantly increase the applicability of legged robots to several mobile manipulation tasks that are not possible for the wheeled or tracked counterparts. The standard hierarchical control pipeline for such legged manipulators is to decouple the controller into that of manipulation and locomotion. However, this is ineffective. It requires immense engineering to support coordination between the arm and legs, and error can propagate across modules causing non-smooth unnatural motions. It is also biological implausible given evidence for strong motor synergies across limbs.…
We present an open-source Visual-Inertial-Leg Odometry (VILO) state estimation solution, Cerberus, for legged robots that estimates position precisely on various terrains in real time using a set of standard sensors, including stereo cameras, IMU, joint encoders, and contact sensors. In addition to estimating robot states, we also perform online kinematic parameter calibration and contact outlier rejection to substantially reduce position drift. Hardware experiments in various indoor and outdoor environments validate that calibrating kinematic parameters within the Cerberus can reduce estimati…
REASAN: Learning Reactive Safe Navigation for Legged Robots
Bernoulli Institute for Mathematics, Computer Science and Artificial Intelligence, University of Groningen, The Netherlands（格罗宁根大学伯努利数学、计算机科学与人工智能研究所）；Department of Electrical Engineering, Linköping University, Sweden（利厄普大学电气工程系）；Engineering and Technology Institute Groningen, University of Groningen, The Netherlands（格罗宁根大学格罗宁根工程与技术研究所）
ROBEL is an open-source platform of cost-effective robots designed for reinforcement learning in the real world. ROBEL introduces two robots, each aimed to accelerate reinforcement learning research in different task domains: D'Claw is a three-fingered hand robot that facilitates learning dexterous manipulation tasks, and D'Kitty is a four-legged robot that facilitates learning agile legged locomotion tasks. These low-cost, modular robots are easy to maintain and are robust enough to sustain on-hardware reinforcement learning from scratch with over 14000 training hours registered on them to da…
CSIRO（联邦科学与工业研究组织）；University of Queensland（昆士兰大学）
We combine MAP-Elites and highly parallelisable simulation to explore the design space of a class of large legged robots, which stand at around 2m tall and whose design and construction is not well-studied. The simulation is modified to account for factors such as motor torque and weight, and presents a reasonable fidelity search space. A novel robot encoding allows for bio-inspired features such as legs scaling along the length of the body. The impact of three possible control generation schemes are assessed in the context of body-brain co-evolution, showing that even constrained problems ben…
We present a method for efficient learning of control policies for multiple related robotic motor skills. Our approach consists of two stages, joint training and specialization training. During the joint training stage, a neural network policy is trained with minimal information to disambiguate the motor skills. This forces the policy to learn a common representation of the different tasks. Then, during the specialization training stage we selectively split the weights of the policy based on a per-weight metric that measures the disagreement among the multiple tasks. By splitting part of the c…
Teaching Robots Like Dogs: Learning Agile Navigation from Luring, Gesture, and Speech
Department of Artificial Intelligence, Korea University（人工智能系，韩国大学）；Department of Computer Science, ETH Zurich（计算机科学系，苏黎世联邦理工学院）；Department of Mechanical and Aerospace Engineering, UCLA（机械与航空航天工程系，加州大学洛杉矶分校）
Animals are capable of precise and agile locomotion using vision. Replicating this ability has been a long-standing goal in robotics. The traditional approach has been to decompose this problem into elevation mapping and foothold planning phases. The elevation mapping, however, is susceptible to failure and large noise artifacts, requires specialized hardware, and is biologically implausible. In this paper, we present the first end-to-end locomotion system capable of traversing stairs, curbs, stepping stones, and gaps. We show this result on a medium-sized quadruped robot using a single front-…
Robots with Attitude: Singularity-Free Quaternion-Based Model-Predictive Control for Agile Legged Robots
Center for Robotics and Biosystems, Northwestern University（西北大学机器人与生物系统中心）；Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）；Department of Mechanical Engineering, Carnegie Mellon University（卡内基梅隆大学机械工程系）
KYON: Semi-Modular Wheel-Legged Quadruped With Agile Bimanual Capability
Humanoids and Human Centered Mechatronics, Fondazione Istituto Italiano di Tecnologia（人机一体化机械系统，意大利理工研究院）；Advanced Robotics Facility, Fondazione Istituto Italiano di Tecnologia（先进机器人设施，意大利理工研究院）
Nowadays, design and development of legged quadruped robots is a quite active area of scientific research. In fact, the legged robots have become popular due to their capabilities to adapt to harsh terrains and diverse environmental conditions in comparison to other mobile robots. With the higher demand for legged robot experiments, more researches and engineers need an affordable and quick way of locomotion algorithm development. In this paper, we present a new open source quadruped robot HyperDog platform, which features 12 RC servo motors, onboard NVIDIA Jetson nano computer and STM32F4 Dis…
