Legged robots often use separate control policiesthat are highly engineered for traversing difficult terrain suchas stairs, gaps, and steps, where switching between policies isonly possible when the robot is in a region that is commonto adjacent controllers. Deep Reinforcement Learning (DRL)is a promising alternative to hand-crafted control design,though typically requires the full set of test conditions to beknown before training. DRL policies can result in complex(often unrealistic) behaviours that have few or no overlappingregions between adjacent policies, making it difficult to switchbehaviours. In this work we develop multiple DRL policieswith Curriculum Learning (CL), each that can traverse asingle respective terrain condition, while ensuring an overlapbetween policies. We then train a network for each destinationpolicy that estimates the likelihood of successfully switchingfrom any other policy. We evaluate our switching methodon a previously unseen combination of terrain artifacts andshow that it performs better than heuristic methods. Whileour method is trained on individual terrain types, it performscomparably to a Deep Q Network trained on the full set ofterrain conditions. This approach allows the development ofseparate policies in constrained conditions with embedded priorknowledge about each behaviour, that is scalable to any numberof behaviours, and prepares DRL methods for applications inthe real world
Terrain-Aware Quadrupedal Locomotion via Reinforcement Learning
Tencent Robotics X（腾讯Robotics X）；The Chinese University of Hong Kong（香港中文大学）；Southern University of Science and Technology（南方科技大学）；University of Alberta（阿尔伯塔大学）
Queensland University of Technology（昆士兰科技大学）；CSIRO（联邦科学与工业研究组织）；Monash University（莫纳什大学）
Reliable bipedal walking over complex terrain is a challenging problem, using a curriculum can help learning. Curriculum learning is the idea of starting with an achievable version of a task and increasing the difficulty as a success criteria is met. We propose a 3-stage curriculum to train Deep Reinforcement Learning policies for bipedal walking over various challenging terrains. In the first stage, the agent starts on an easy terrain and the terrain difficulty is gradually increased, while forces derived from a target policy are applied to the robot joints and the base. In the second stage, …
Umeå University（于默奥大学）；Swedish University of Agricultural Sciences（瑞典农业科学大学）
We explore the potential to control terrain vehicles using deep reinforcement in scenarios where human operators and traditional control methods are inadequate. This letter presents a controller that perceives, plans, and successfully controls a 16-tonne forestry vehicle with two frame articulation joints, six wheels, and their actively articulated suspensions to traverse rough terrain. The carefully shaped reward signal promotes safe, environmental, and efficient driving, which leads to the emergence of unprecedented driving skills. We test learned skills in a virtual environment, including t…
Dynamic quadruped locomotion over challenging terrains with precise foot placements is a hard problem for both optimal control methods and Reinforcement Learning (RL). Non-linear solvers can produce coordinated constraint satisfying motions, but often take too long to converge for online application. RL methods can learn dynamic reactive controllers but require carefully tuned shaping rewards to produce good gaits and can have trouble discovering precise coordinated movements. Imitation learning circumvents this problem and has been used with motion capture data to extract quadruped gaits for …
This paper presents a deep learning framework that is capable of solving partially observable locomotion tasks based on our novel interpretation of Recurrent Deterministic Policy Gradient (RDPG). We study on bias of sampled error measure and its variance induced by the partial observability of environment and subtrajectory sampling, respectively. Three major improvements are introduced in our RDPG based learning framework: tail-step bootstrap of interpolated temporal difference, initialisation of hidden state using past trajectory scanning, and injection of external experiences learned by othe…
Istituto Italiano di Tecnologia（意大利理工学院）；University of Oxford（牛津大学）；LAAS-CNRS（法国国家科学研究中心自动化系统实验室）
The quality of the visual feedback can vary significantly on a legged robot that is meant to traverse unknown and unstructured terrains. The map of the environment, acquired with online state-of-the-art algorithms, often degrades after a few steps, due to sensing inaccuracies, slippage and unexpected disturbances. When designing locomotion algorithms, this degradation can result in planned trajectories that are not consistent with the reality, if not dealt properly. In this work, we propose a heuristic-based planning approach that enables a quadruped robot to successfully traverse a significan…
Legged robots are popular candidates for missions in challenging terrains due to the wide variety of locomotion strategies they can employ. Terrain classification is a key enabling technology for autonomous legged robots, as it allows the robot to harness their innate flexibility to adapt their behaviour to the demands of their operating environment. In this paper, we show how highly capable machine learning techniques, namely gated recurrent neural networks, allow our target legged robot to correctly classify the terrain it traverses in both supervised and semi-supervised fashions. Tests on a…
Dynamically Extensible and Retractable Robotic Leg Linkages for Multi-task Execution in Search and Rescue Scenarios
Mechanical and Aerospace Engineering Department, University of California, San Diego（加州大学圣地亚哥分校机械与航空航天工程系）；Electrical and Computer Engineering Department, University of California, San Diego（加州大学圣地亚哥分校电气与计算机工程系）
Shanghai Jiao Tong University（上海交通大学）；Digital Brain Lab（数字大脑实验室）；ShanghaiTech University（上海科技大学）；University College London（伦敦大学学院）
Deep reinforcement learning has recently emerged as an appealing alternative for legged locomotion over multiple terrains by training a policy in physical simulation and then transferring it to the real world (i.e., sim-to-real transfer). Despite considerable progress, the capacity and scalability of traditional neural networks are still limited, which may hinder their applications in more complex environments. In contrast, the Transformer architecture has shown its superiority in a wide range of large-scale sequence modeling tasks, including natural language processing and decision-making pro…
Queensland University of Technology（昆士兰科技大学）；CSIRO（联邦科学与工业研究组织）；Amazon Robotics AI（亚马逊机器人人工智能）；Deakin University（迪肯大学）；Monash University（莫纳什大学）；LYRO Robotics Pty Ltd（LYRO机器人私人有限公司）
Dynamic platforms that operate over many unique terrain conditions typically require many behaviours. To transition safely, there must be an overlap of states between adjacent controllers. We develop a novel method for training setup policies that bridge the trajectories between pre-trained Deep Reinforcement Learning (DRL) policies. We demonstrate our method with a simulated biped traversing a difficult jump terrain, where a single policy fails to learn the task, and switching between pre-trained policies without setup policies also fails. We perform an ablation of key components of our syste…
Google Brain Robotics（谷歌大脑机器人）；Georgia Institute of Technology（佐治亚理工学院）
Legged robots have unparalleled mobility on unstructured terrains. However, it remains an open challenge to design locomotion controllers that can operate in a large variety of environments. In this paper, we address this challenge of automatically learning locomotion controllers that can generalize to a diverse collection of terrains often encountered in the real world. We frame this challenge as a multi-task reinforcement learning problem and define each task as a type of terrain that the robot needs to traverse. We propose an end-to-end learning approach that makes direct use of the raw ext…
University of Freiburg（弗莱堡大学）；Toyota Research Institute（丰田研究院）
Mobile robots operating in unknown urban environments encounter a wide range of complex terrains to which they must adapt their planned trajectory for safe and efficient navigation. Most existing approaches utilize supervised learning to classify terrains from either an exteroceptive or a proprioceptive sensor modality. However, this requires a tremendous amount of manual labeling effort for each newly encountered terrain as well as for variations of terrains caused by changing environmental conditions. In this work, we propose a novel terrain classification framework leveraging an unsupervise…
Field Calibration of Hyperspectral Cameras for Terrain Inference
Electrical and Computer Engineering Department, Northeastern University（电气与计算机工程系，东北大学）；Lincoln Laboratory, Massachusetts Institute of Technology（林肯实验室，麻省理工学院）
Shenzhen Institute of Artificial Intelligence and Robotics for Society (AIRS)（深圳市人工智能与机器人研究院）；Institute of Robotics and Intelligent Manufacturing (IRIM), The Chinese University of Hong Kong (CUHK)（香港中文大学机器人与智能制造研究院）；George W. Woodruff School of Mechanical Engineering, Georgia Institute of Technology（佐治亚理工学院乔治·W·伍德拉夫机械工程学院）；Beijing Institute for General Artificial Intelligence（北京通用人工智能研究院）
Dynamic quadrupedal locomotion over rough terrains reveals remarkable progress over the last few decades. Small-scale quadruped robots are adequately flexible and adaptable to traverse uneven terrains along sagittal direction, such as slopes and stairs. To accomplish autonomous locomotion navigation in complex environments, spinning is a fundamental yet indispensable functionality for legged robots. However, spinning behaviors of quadruped robots on uneven terrain often exhibit position drifts. Motivated by this problem, this study presents an algorithmic method to enable accurate spinning mot…
Animals are capable of precise and agile locomotion using vision. Replicating this ability has been a long-standing goal in robotics. The traditional approach has been to decompose this problem into elevation mapping and foothold planning phases. The elevation mapping, however, is susceptible to failure and large noise artifacts, requires specialized hardware, and is biologically implausible. In this paper, we present the first end-to-end locomotion system capable of traversing stairs, curbs, stepping stones, and gaps. We show this result on a medium-sized quadruped robot using a single front-…
Massachusetts Institute of Technology（麻省理工学院）；robonity: worldwide tech consulting（robonity：全球技术咨询）
Terramechanics plays a critical role in the areas of ground vehicles and ground mobile robots since understanding and estimating the variables influencing the vehicle-terrain interaction may mean the success or the failure of an entire mission. This research applies state-of-the-art algorithms in deep learning to two key problems: estimating wheel slip and classifying the terrain being traversed by a ground robot. Three data sets collected by ground robotic platforms (MIT single-wheel testbed, MSL Curiosity rover, and tracked robot Fitorobot) are employed in order to compare the performance of…
The semantics of the environment, such as the terrain type and property, reveals important information for legged robots to adjust their behaviors. In this work, we present a framework that learns semantics-aware locomotion skills from perception for quadrupedal robots, such that the robot can traverse through complex offroad terrains with appropriate speeds and gaits using perception information. Due to the lack of high-fidelity outdoor simulation, our framework needs to be trained directly in the real world, which brings unique challenges in data efficiency and safety. To ensure sample effic…
Soft pneumatic legged robots show promise in their ability to traverse a range of different types of terrain, including natural unstructured terrain met in applications like precision agriculture. They can adapt their body morphology to the intricacies of the terrain at hand, thus enabling robust and resilient locomotion. In this paper we capitalize upon recent developments on soft pneumatic legged robots to introduce a closed-loop trajectory tracking control scheme for operation over flat ground. Closed-loop pneumatic actuation feedback is achieved via a compact and portable pneumatic regulat…
PIE: Parkour with Implicit-Explicit Learning Framework for Legged Robots
Institute of Cyber-Systems and Control, Zhejiang University（浙江大学控制科学与工程学院 cyber-system 研究所）；State Key Laboratory of Industrial Control Technology（工业控制技术国家重点实验室）
Evolutionary Morphology Towards Overconstrained Locomotion via Large-Scale, Multi-Terrain Deep Reinforcement Learning
Southern University of Science and Technology（南方科技大学）；School of Design（设计学院）；Department of Mechanical and Energy Engineering（机械与能源工程系）；Department of System Design and Intelligent Manufacturing（系统设计与智能制造系）
As mobile robots become useful performing everyday tasks in complex real-world environments, they must be able to traverse a range of difficult terrain types such as stairs, stepping stones, gaps, jumps and narrow passages. This work investigated traversing these types of environments with a bipedal robot (simulation experiments), and a tracked robot (real world). Developing a traditional monolithic controller for traversing all terrain types is challenging, and for large physical robots realistic test facilities are required and safety must be ensured. An alternative is a suite of simple beha…
The recent availability of commercial-off-the-shelf (COTS) legged robot platforms have opened up new opportunities in deploying legged systems into different scenarios. While the main advantage of legged robots is their ability to traverse unstructured terrain, there are still large gaps between what robot platforms can achieve and their animal counterparts. Therefore, when deploying as part of a heterogeneous robot team of different platforms, it is beneficial to understand the different scenarios where a legged platform would perform better than a wheeled, tracked or aerial platform. Two COT…
NASA Jet Propulsion Laboratory, California Institute of Technology（美国国家航空航天局喷气推进实验室，加州理工学院）；University of Goettingen, University Medical Center Goettingen（哥廷根大学，哥廷根大学医学中心）；Institute for Robotics and Intelligent Machines, Georgia Institute of Technology（佐治亚理工学院机器人与智能机器研究所）；Swiss Federal Institute of Technology (ETH Zürich)（苏黎世联邦理工学院）；California Institute of Technology（加州理工学院）
Legged robots can traverse a wide variety of terrains, some of which may be challenging for wheeled robots, such as stairs or highly uneven surfaces. However, quadruped robots face stability challenges on slippery surfaces. This can be resolved by adjusting the robot's locomotion by switching to more conservative and stable locomotion modes, such as crawl mode (where three feet are in contact with the ground always) or amble mode (where one foot touches down at a time) to prevent potential falls. To tackle these challenges, we propose an approach to learn a model from past robot experience for…
FZI Research Center for Information Technology（弗劳恩霍夫研究所信息技术研究中心）；Machine Intelligence and Robotics Lab at the Karlsruhe Institute for Technology (KIT)（卡尔斯鲁厄理工学院机械智能与机器人实验室）
Robots "in-the-wild" encounter and must traverse widely varying terrain, ranging from solid ground to granular materials like sand to full liquids. Numerous approaches exist, including wheeled and legged robots, each excelling in specific domains. Screw-based locomotion is a promising approach for multi-domain mobility, leveraged in exploratory robotic designs, including amphibious vehicles and snake robotics. However, unlike other forms of locomotion, there is a limited exploration of the models, parameter effects, and efficiency for multi-terrain Archimedes screw locomotion. In this work, we…
LIMBERO: A Limbed Climbing Exploration Robot Toward Traveling on Rocky Cliffs
Space Robotics Lab. (SRL) in Department of Aerospace Engineering, Graduate School of Engineering, Tohoku University（东京东京大学工学研究院航空航天工程系空间机器人实验室）；National Institute of Applied Sciences of Toulouse（图卢兹应用科学国家研究院）
SlipSense: Multimodal Sensing for Online Slip Detection in Legged Robots
School of Electrical and Electronic Engineering, Nanyang Technological University（南洋理工大学电气与电子工程学院）；Institute for Infocomm Research, Agency for Science Technology and Research（资讯通信研究院，科技研究局）
University of Texas at Arlington（德克萨斯大学阿灵顿分校）；NCSR Demokritos（希腊国家科学研究中心德谟克利特）
Traversability illustrates the difficulty of driving through a specific region and encompasses the suitability of the terrain for traverse based on its physical properties, such as slope and roughness, surface condition, etc. In this survey we highlight the merits and limitations of all the major steps in the evolution of traversability estimation techniques, covering both non-trainable and machine-learning methods, leading up to the recent proliferation of deep learning literature. We discuss how the nascence of Deep Learning has created an opportunity for radical improvement in traversabilit…
A terrestrial robot that can maneuver rough terrain and scout places is very useful in mapping out unknown areas. It can also be used explore dangerous areas in place of humans. A terrestrial robot modeled after a scorpion will be able to traverse undetected and can be used for surveillance purposes. Therefore, this paper proposes modelling of a scorpion inspired robot and a reinforcement learning (RL) based controller for navigation. The robot scorpion uses serial four bar mechanisms for the legs movements. It also has an active tail and a movable claw. The controller is trained to navigate t…
Peking University（北京大学）；China North Vehicle Research Institute（中国北方车辆研究所）
Road detection or traversability analysis has been a key technique for a mobile robot to traverse complex off-road scenes. The problem has been mainly formulated in early works as a binary classification one, e.g. associating pixels with road or non-road labels. Whereas understanding scenes with fine-grained labels are needed for off-road robots, as scenes are very diverse, and the various mechanical performance of off-road robots may lead to different definitions of safe regions to traverse. How to define and annotate fine-grained labels to achieve meaningful scene understanding for a robot t…
University of Oslo（奥斯陆大学）；Norwegian Defence Research Institute（挪威国防研究所）
Legged robots promise higher versatility and the ability to traverse much more difficult terrains than their wheeled counterparts. Even though the use of legged robots have increased drastically in the last few years, they are still not close to the performance seen from legged animals in nature. Robotic legs are typically fairly simple mechanically, and few feature an ankle joint, even though most land mammals have one. The ankle could be a key to better performance and stability for legged robots, and in this paper we investigate how the use of an ankle in a bipedal robot could improve its p…
Colorado School of Mines（科罗拉多矿业学院）；US Army Research Laboratory（美国陆军研究实验室）
Terrain adaptation is an essential capability for a ground robot to effectively traverse unstructured off-road terrain in real-world field environments such as forests. However, the expected robot behaviors generated by terrain adaptation methods cannot always be executed accurately due to setbacks such as wheel slip and reduced tire pressure. To address this problem, we propose a novel approach for consistent behavior generation that enables the ground robot's actual behaviors to more accurately match expected behaviors while adapting to a variety of unstructured off-road terrain. Our approac…
Legged Autonomous Surface Science In Analogue Environments (LASSIE): Making Every Robotic Step Count in Planetary Exploration
Collaborative Robotics and Intelligent Systems Institute, Oregon State University（协作机器人与智能系统研究所，俄勒冈州立大学）；Department of Geology and Geophysics, Texas A&M University（地质与地球物理系，德克萨斯大学）；Department of Electrical and Computer Engineering, University of Southern California（电气与计算机工程系，南加州大学）；Department of Earth and Environmental Science, University of Pennsylvania（地球与环境科学系，宾夕法尼亚大学）；Department of Astronomy and Planetary Science, Northern Arizona University（天文学与行星科学系，北方亚利桑那大学）；School of Earth and Atmospheric Sciences, Georgia Institute of Technology（地球与大气科学学院，佐治亚理工学院）；NASA Marshall Space Flight Center（美国国家航空航天局马歇尔航天飞行中心）；NASA Johnson Space Center（美国国家航空航天局约翰逊航天中心）；Department of Electrical and Systems Engineering, University of Pennsylvania（电气与系统工程系，宾夕法尼亚大学）；Department of Psychology, Temple University（心理学系， Temple 大学）
MoRE: Mixture of Residual Experts for Humanoid Lifelike Gaits Learning on Complex Terrains
University of Science and Technology of China（科学技术大学）；Institute of Artificial Intelligence (TeleAI), China Telecom（人工智能研究所（TeleAI））；Harbin Engineering University（哈尔滨工程大学）；ShanghaiTech University（上海科技大学）
Scientific Exploration of Challenging Planetary Analog Environments with a Team of Legged Robots
ETH Zurich（苏黎世联邦理工学院）；Harbin Institute of Technology（哈尔滨工业大学）；University of Bern（伯尔尼大学）；University of Basel（巴塞尔大学）；University of Zurich（苏黎世大学）；Lucerne University of Applied Sciences and Art（卢塞恩应用科学与艺术大学）
