Legged robots have unparalleled mobility on unstructured terrains. However, it remains an open challenge to design locomotion controllers that can operate in a large variety of environments. In this paper, we address this challenge of automatically learning locomotion controllers that can generalize to a diverse collection of terrains often encountered in the real world. We frame this challenge as a multi-task reinforcement learning problem and define each task as a type of terrain that the robot needs to traverse. We propose an end-to-end learning approach that makes direct use of the raw exteroceptive inputs gathered from a simulated 3D LiDAR sensor, thus circumventing the need for ground-truth heightmaps or preprocessing of perception information. As a result, the learned controller demonstrates excellent zero-shot generalization capabilities and can navigate 13 different environments, including stairs, rugged land, cluttered offices, and indoor spaces with humans.
Shenzhen Institute of Artificial Intelligence and Robotics for Society (AIRS)（深圳市人工智能与机器人研究院）；Institute of Robotics and Intelligent Manufacturing (IRIM), The Chinese University of Hong Kong (CUHK)（香港中文大学机器人与智能制造研究院）；George W. Woodruff School of Mechanical Engineering, Georgia Institute of Technology（佐治亚理工学院乔治·W·伍德拉夫机械工程学院）；Beijing Institute for General Artificial Intelligence（北京通用人工智能研究院）
Dynamic quadrupedal locomotion over rough terrains reveals remarkable progress over the last few decades. Small-scale quadruped robots are adequately flexible and adaptable to traverse uneven terrains along sagittal direction, such as slopes and stairs. To accomplish autonomous locomotion navigation in complex environments, spinning is a fundamental yet indispensable functionality for legged robots. However, spinning behaviors of quadruped robots on uneven terrain often exhibit position drifts. Motivated by this problem, this study presents an algorithmic method to enable accurate spinning mot…
Istituto Italiano di Tecnologia（意大利技术研究院）；Università degli Studi di Genova（热那亚大学）；Swiss Federal Institute of Technology at Lausanne（洛桑联邦理工学院）；University of Texas（德克萨斯大学）；University of Naples Federico II（那不勒斯费德里科二世大学）
(Simplified Abstract) To accomplish breakthroughs in dynamic whole-body locomotion, legged robots have to be terrain aware. Terrain-Aware Locomotion (TAL) implies that the robot can perceive the terrain with its sensors, and can take decisions based on this information. This thesis presents TAL strategies both from a proprioceptive and an exteroceptive perspective. The strategies are implemented at the level of locomotion planning, control, and state estimation, and using optimization and learning techniques. The first part is on TAL strategies at the Whole-Body Control (WBC) level. We introdu…
Optimizing gait stability for legged robots is a difficult problem. Even on level surfaces, effectively traversing across different textures (e.g., carpet) rests on dynamically tuning parameters in multidimensional space. Inspired by biology, evolutionary algorithms (EA) remain an attractive solution for feasibly implementing robotic locomotion with both energetic economy and rapid parameter convergence. Here, we leveraged this class of algorithms to evolve a stable hexapod gait controller capable of traversing uneven terrain and obstacles. Gait parameters were evolved in a rigid body dynamics…
Legged robots are becoming increasingly powerful and popular in recent years for their potential to bring the mobility of autonomous agents to the next level. This work presents a deep reinforcement learning approach that learns a robust Lidar-based perceptual locomotion policy in a partially observable environment using Proximal Policy Optimisation. Visual perception is critical to actively overcome challenging terrains, and to do so, we propose a novel learning strategy: Dynamic Reward Strategy (DRS), which serves as effective heuristics to learn a versatile gait using a neural network archi…
Terrain-Aware Quadrupedal Locomotion via Reinforcement Learning
Tencent Robotics X（腾讯Robotics X）；The Chinese University of Hong Kong（香港中文大学）；Southern University of Science and Technology（南方科技大学）；University of Alberta（阿尔伯塔大学）
This work is on vision-based planning strategies for legged robots that separate locomotion planning into foothold selection and pose adaptation. Current pose adaptation strategies optimize the robot's body pose relative to given footholds. If these footholds are not reached, the robot may end up in a state with no reachable safe footholds. Therefore, we present a Vision-Based Terrain-Aware Locomotion (ViTAL) strategy that consists of novel pose adaptation and foothold selection algorithms. ViTAL introduces a different paradigm in pose adaptation that does not optimize the body pose relative t…
Legged robots need to be capable of walking on diverse terrain conditions. In this paper, we present a novel reinforcement learning framework for learning locomotion on non-rigid dynamic terrains. Specifically, our framework can generate quadruped locomotion on flat elastic terrain that consists of a matrix of tiles moving up and down passively when pushed by the robot's feet. A trained robot with 55cm base length can walk on terrain that can sink up to 5cm. We propose a set of observation and reward terms that enable this locomotion; in which we found that it is crucial to include the end-eff…
Look Forward to Walk Backward: Efficient Terrain Memory for Backward Locomotion with Forward Vision
Institute of Cyber-Systems and Control, Zhejiang University（浙江大学控制系统研究所）；College of Artificial Intelligence, Nankai University（南开大学人工智能学院）；Hangzhou Public Library（杭州公共图书馆）
Robots "in-the-wild" encounter and must traverse widely varying terrain, ranging from solid ground to granular materials like sand to full liquids. Numerous approaches exist, including wheeled and legged robots, each excelling in specific domains. Screw-based locomotion is a promising approach for multi-domain mobility, leveraged in exploratory robotic designs, including amphibious vehicles and snake robotics. However, unlike other forms of locomotion, there is a limited exploration of the models, parameter effects, and efficiency for multi-terrain Archimedes screw locomotion. In this work, we…
ATRos: Learning Energy-Efficient Agile Locomotion for Wheeled-legged Robots
Shanghai Huawei Technologies Co., Ltd.（上海华为技术有限公司）；College of Future Information Technology, Fudan University（复旦大学未来信息科技学院）；Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）；School of Aeronautic and Astronautics, Zhejiang University（浙江大学航空宇航学院）；School of Electronics and Communication Engineering, Sun Yat-sen University（中山大学电子与通信工程学院）
Evolutionary Morphology Towards Overconstrained Locomotion via Large-Scale, Multi-Terrain Deep Reinforcement Learning
Southern University of Science and Technology（南方科技大学）；School of Design（设计学院）；Department of Mechanical and Energy Engineering（机械与能源工程系）；Department of System Design and Intelligent Manufacturing（系统设计与智能制造系）
Despite many accomplishments by legged robot designers, state-of-the-art bipedal robots are prone to falling over, cannot negotiate extremely rough terrains and cannot directly regulate unilateral contact forces. Our objective is to integrate merits of legged and aerial robots in a single platform. We will show that the thrusters in a bipedal legged robot called Harpy can be leveraged to stabilize the robot's frontal dynamics and permit jumping over large obstacles which is an unusual capability not reported before. In addition, we will capitalize on the thrusters action in Harpy and will show…
We propose a learning-based method to reconstruct the local terrain for locomotion with a mobile robot traversing urban environments. Using a stream of depth measurements from the onboard cameras and the robot's trajectory, the algorithm estimates the topography in the robot's vicinity. The raw measurements from these cameras are noisy and only provide partial and occluded observations that in many cases do not show the terrain the robot stands on. Therefore, we propose a 3D reconstruction model that faithfully reconstructs the scene, despite the noisy measurements and large amounts of missing…
The semantics of the environment, such as the terrain type and property, reveals important information for legged robots to adjust their behaviors. In this work, we present a framework that learns semantics-aware locomotion skills from perception for quadrupedal robots, such that the robot can traverse through complex offroad terrains with appropriate speeds and gaits using perception information. Due to the lack of high-fidelity outdoor simulation, our framework needs to be trained directly in the real world, which brings unique challenges in data efficiency and safety. To ensure sample effic…
Legged locomotion is commonly studied and expressed as a discrete set of gait patterns, like walk, trot, gallop, which are usually treated as given and pre-programmed in legged robots for efficient locomotion at different speeds. However, fixing a set of pre-programmed gaits limits the generality of locomotion. Recent animal motor studies show that these conventional gaits are only prevalent in ideal flat terrain conditions while real-world locomotion is unstructured and more like bouts of intermittent steps. What principles could lead to both structured and unstructured patterns across mammal…
Effect of Gait Design on Proprioceptive Sensing of Terrain Properties in a Quadrupedal Robot
Department of Mechanical Engineering, Massachusetts Institute of Technology（机械工程系，麻省理工学院）；Ming Hsieh Department of Electrical and Computer Engineering, University of Southern California（Ming Hsieh 电子与计算机工程系，南加州大学）；Department of Earth and Environmental Science, University of Pennsylvania（地球与环境科学系，宾夕法尼亚大学）
本文对比四足机器人的Crawl N' Sense与Trot-Walk步态，证明慢速爬行步态能更稳定、准确地感知可变形地形强度及表面结壳破裂。
Locomotion over soft terrain remains a challenging problem for legged robots. Most of the work done on state estimation for legged robots is designed for rigid contacts, and does not take into account the physical parameters of the terrain. That said, this letter answers the following questions: how and why does soft terrain affect state estimation for legged robots? To do so, we utilized a state estimator that fuses IMU measurements with leg odometry that is designed with rigid contact assumptions. We experimentally validated the state estimator with the HyQ robot trotting over both soft and …
GM-Loco: Terrain-Adaptive Humanoid Locomotion on Granular Media
Institute for Robotics and Intelligent Machines, Georgia Institute of Technology（佐治亚理工学院机器人与智能机器研究所）；School of Physics, Georgia Institute of Technology（佐治亚理工学院物理学院）；Northeastern University（东北大学）
TAGA: Terrain-aware Active Gaze Learning for Generalizable Agile Humanoid Locomotion
MarmotLab, National University of Singapore（马尔莫实验室，新加坡国立大学）；Center of X-Mechanics, Zhejiang University（浙大X力学中心）；South China University of Technology（华南理工大学）
Institute for Infocomm Research, A*STAR（新加坡科技研究局资讯通信研究院）；University College London（伦敦大学学院）
This work developed a learning framework for perceptive legged locomotion that combines visual feedback, proprioceptive information, and active gait regulation of foot-ground contacts. The perception requires only one forward-facing camera to obtain the heightmap, and the active regulation of gait paces and traveling velocity are realized through our formulation of CPG-based high-level imitation of foot-ground contacts. Through this framework, an end-user has the ability to command task-level inputs to control different walking speeds and gait frequencies according to the traversal of differen…
UniReLo: Learning a Unified Humanoid Policy from Fall Recovery to Locomotion across Diverse Terrains
School of Control Science and Engineering, Shandong University（山东大学控制科学与工程学院）；Key Laboratory of Machine Intelligence and System Control, Ministry of Education（教育部机器智能与系统控制重点实验室）
Load-bearing Assessment for Safe Locomotion of Quadruped Robots on Collapsing Terrain
Dynamic Legged Systems Lab, Istituto Italiano di Tecnologia (IIT)（动态腿系统实验室，意大利技术研究院）；Department of Mechanical Engineering, São Carlos School of Engineering (EESC), University of São Paulo (USP)（机械工程系，圣卡洛斯工程学院（EESC），圣保罗大学）
As mobile robots become useful performing everyday tasks in complex real-world environments, they must be able to traverse a range of difficult terrain types such as stairs, stepping stones, gaps, jumps and narrow passages. This work investigated traversing these types of environments with a bipedal robot (simulation experiments), and a tracked robot (real world). Developing a traditional monolithic controller for traversing all terrain types is challenging, and for large physical robots realistic test facilities are required and safety must be ensured. An alternative is a suite of simple beha…
A Foot Resistive Force Model for Legged Locomotion on Muddy Terrains
Department of Mechanical and Aerospace Engineering, Rutgers University（罗格斯大学机械与航空航天工程系）；Department of Electrical and Biomedical Engineering, University of Nevada（内华达大学电气与生物医学工程系）
ZSL-RPPO: Zero-Shot Learning for Quadrupedal Locomotion in Challenging Terrains using Recurrent Proximal Policy Optimization
Huawei Technologies（华为技术有限公司）；Huawei Munich Research Center（华为慕尼黑研究中心）；University College London（伦敦大学学院）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；East China Normal University（华东师范大学）
