Urban search and rescue missions require rapid first response to minimize loss of life and damage. Often, such efforts are assisted by humanitarian robots which need to handle dynamic operational conditions such as uneven and rough terrains, especially during mass casualty incidents like an earthquake. Quadruped robots, owing to their versatile design, have the potential to assist in such scenarios. However, control of quadruped robots in dynamic and rough terrain environments is a challenging problem due to the many degrees of freedom of these robots. Current locomotion controllers for quadrupeds are limited in their ability to produce multiple adaptive gaits, solve tasks in a time and resource-efficient manner, and require tedious training and manual tuning procedures. To address these challenges, we propose MTAC: a multi-gait terrain-adaptive controller, which utilizes a Hierarchical reinforcement learning (HRL) approach while being time and memory-efficient. We show that our proposed method scales well to a diverse range of environments with similar compute times as state-of-the-art methods. Our method showed greater than 75% on most tasks, outperforming previous work on the majority of test cases.
There is a growing interest in learning a velocity command tracking controller of quadruped robot using reinforcement learning due to its robustness and scalability. However, a single policy, trained end-to-end, usually shows a single gait regardless of the command velocity. This could be a suboptimal solution considering the existence of optimal gait according to the velocity for quadruped animals. In this work, we propose a hierarchical controller for quadruped robot that could generate multiple gaits (i.e. pace, trot, bound) while tracking velocity command. Our controller is composed of two…
Efficient Learning of Control Policies for Robust Quadruped Bounding using Pretrained Neural Networks
Institute of Cyber-Systems and Control, Zhejiang University（浙江大学控制科学与工程学院）；Intelligent Robot Research Center, Zhejiang Lab（之江实验室智能机器人研究中心）；Department of Computer Science, University College London（伦敦大学学院计算机科学系）；State Key Laboratory of Industrial Control Technology, Zhejiang University（浙江大学工业控制技术国家重点实验室）
Chinese University of Hong Kong（香港中文大学）；Baidu Inc.（百度公司）；Southern University of Science and Technology（南方科技大学）；Shenzhen Research Institute of the Chinese University of Hong Kong（香港中文大学深圳研究院）
Recently reinforcement learning (RL) has emerged as a promising approach for quadrupedal locomotion, which can save the manual effort in conventional approaches such as designing skill-specific controllers. However, due to the complex nonlinear dynamics in quadrupedal robots and reward sparsity, it is still difficult for RL to learn effective gaits from scratch, especially in challenging tasks such as walking over the balance beam. To alleviate such difficulty, we propose a novel RL-based approach that contains an evolutionary foot trajectory generator. Unlike prior methods that use a fixed tr…
Zhejiang University（浙江大学）；ZJU-Hangzhou Global Scientific and Technological Innovation Center（浙江大学-杭州市全球科技创新中心）；State Key Laboratory of Fluid Power and Mechatronic System, Zhejiang University（浙江大学流体动力与机电系统国家重点实验室）；Institute of Applied Mechanics, Zhejiang University（浙江大学应用力学研究所）
Gaits and transitions are key components in legged locomotion. For legged robots, describing and reproducing gaits as well as transitions remain longstanding challenges. Reinforcement learning has become a powerful tool to formulate controllers for legged robots. Learning multiple gaits and transitions, nevertheless, is related to the multi-task learning problems. In this work, we present a novel framework for training a simple control policy for a quadruped robot to locomote in various gaits. Four independent phases are used as the interface between the gait generator and the control policy, …
Ecole Polytechnique Federale de Lausanne（洛桑联邦理工学院）
In this letter, we present a method for integrating central pattern generators (CPGs), i.e. systems of coupled oscillators, into the deep reinforcement learning (DRL) framework to produce robust and omnidirectional quadruped locomotion. The agent learns to directly modulate the intrinsic oscillator setpoints (amplitude and frequency) and coordinate rhythmic behavior among different oscillators. This approach also allows the use of DRL to explore questions related to neuroscience, namely the role of descending pathways, interoscillator couplings, and sensory feedback in gait generation. We trai…
Learning Gait-Aware Quadruped Locomotion with Temporal Logic Specifications
Department of Electrical Engineering and Computer Sciences University of California Berkeley（加州大学伯克利分校电气工程与计算机科学系）；University of Southern California（南加州大学）
This paper systematically decomposes a quadrupedal robot into bipeds to rapidly generate walking gaits and then recomposes these gaits to obtain quadrupedal locomotion. We begin by decomposing the full-order, nonlinear and hybrid dynamics of a three-dimensional quadrupedal robot, including its continuous and discrete dynamics, into two bipedal systems that are subject to external forces. Using the hybrid zero dynamics (HZD) framework, gaits for these bipedal robots can be rapidly generated (on the order of seconds) along with corresponding controllers. The decomposition is achieved in such a w…
It is very important for quadruped walking machine to keep its stability in high speed walking. It has been indicated that moment around the supporting diagonal line of quadruped in trotting gait largely influences walking stability. In this paper, moment around the supporting diagonal line of quadruped in trotting gait is modeled and its effects on body attitude are analyzed. The degree of influence varies with different initial stances of quadruped and we get the optimal initial stance of quadruped in trotting gait with maximal walking stability. Simulation results are presented. Keywords: q…
It is often overlooked by roboticists when designing locomotion controllers for their legged machines, that energy consumption plays an important role in selecting the best gaits for locomotion at high speeds or over long distances. The purpose of this study is to examine four similar asymmetrical quadrupedal gaits that are frequently observed in legged animals in nature. To understand how a specific footfall pattern will change the energetics of a legged system, we first developed a full body model of a quadrupedal robot called A1. And for each gait we created a hybrid system with desired foo…
Designing agile locomotion for quadruped robots often requires extensive expertise and tedious manual tuning. In this paper, we present a system to automate this process by leveraging deep reinforcement learning techniques. Our system can learn quadruped locomotion from scratch using simple reward signals. In addition, users can provide an open loop reference to guide the learning process when more control over the learned gait is needed. The control policies are learned in a physics simulator and then deployed on real robots. In robotics, policies trained in simulation often do not transfer t…
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Vanderbilt University（范德堡大学）
We tackle the problem of perceptive locomotion in dynamic environments. In this problem, a quadrupedal robot must exhibit robust and agile walking behaviors in response to environmental clutter and moving obstacles. We present a hierarchical learning framework, named PRELUDE, which decomposes the problem of perceptive locomotion into high-level decision-making to predict navigation commands and low-level gait generation to realize the target commands. In this framework, we train the high-level navigation controller with imitation learning on human demonstrations collected on a steerable cart a…
Princeton University（普林斯顿大学）；Google Research（谷歌研究院）；Georgia Institute of Technology（佐治亚理工学院）
Designing control policies for legged locomotion is complex due to the under-actuated and non-continuous robot dynamics. Model-free reinforcement learning provides promising tools to tackle this challenge. However, a major bottleneck of applying model-free reinforcement learning in real world is safety. In this paper, we propose a safe reinforcement learning framework that switches between a safe recovery policy that prevents the robot from entering unsafe states, and a learner policy that is optimized to complete the task. The safe recovery policy takes over the control when the learner polic…
Synthesizing a stable gait that enables a quadruped robot to climb stairs is the focus of this paper. To this end, first a stable transition from initial to desired configuration is made based on the minimum number of steps and maximum use of the leg workspace to prepare the robot for the movement. Next, swing leg and body trajectories are planned for a successful stair- climbing gait. Afterwards, a stable spinning gait is proposed to change the orientation of the body. We simulate our gait planning algorithms on a model of quadruped robot. The results show that the robot is able to climb up s…
United States Military Academy（美国军事学院）；Massachusetts Institute of Technology（麻省理工学院）
The Extra Robotic Legs (XRL) system is a robotic augmentation worn by a human operator consisting of two articulated robot legs that walk with the operator and help bear a heavy backpack payload. It is desirable for the Human-XRL quadruped system to walk with the rear legs lead the front by 25% of the gait period, minimizing the energy lost from foot impacts while maximizing balance stability. Unlike quadrupedal robots, the XRL cannot command the human's limbs to coordinate quadrupedal locomotion. Using a pair of Rimless Wheel models, it is shown that the systems coupled with a spring and damp…
We present a learning algorithm for training a single policy that imitates multiple gaits of a walking robot. To achieve this, we use and extend MPC-Net, which is an Imitation Learning approach guided by Model Predictive Control (MPC). The strategy of MPC-Net differs from many other approaches since its objective is to minimize the control Hamiltonian, which derives from the principle of optimality. To represent the policies, we employ a mixture-of-experts network (MEN) and observe that the performance of a policy improves if each expert of a MEN specializes in controlling exactly one mode of …
In this work we present a trajectory Optimization framework for whole-body motion planning through contacts. We demonstrate how the proposed approach can be applied to automatically discover different gaits and dynamic motions on a quadruped robot. In contrast to most previous methods, we do not pre-specify contact switches, timings, points or gait patterns, but they are a direct outcome of the optimization. Furthermore, we optimize over the entire dynamics of the robot, which enables the optimizer to fully leverage the capabilities of the robot. To illustrate the spectrum of achievable motion…
During learning trials, systems are exposed to different failure conditions which may break robotic parts before a safe behavior is discovered. Humans contour this problem by grounding their learning to a safer structure/control first and gradually increasing its difficulty. This paper presents the impact of a similar supports in the learning of a stable gait on a quadruped robot. Based on the psychological theory of instructional scaffolding, we provide different support settings to our robot, evaluated with strain gauges, and use Bayesian Optimization to conduct a parametric search towards a…
Oxford Robotics Institute, University of Oxford（牛津大学牛津机器人研究所）
Quadruped locomotion is rapidly maturing to a degree where robots now routinely traverse a variety of unstructured terrains. However, while gaits can be varied typically by selecting from a range of pre-computed styles, current planners are unable to vary key gait parameters continuously while the robot is in motion. The synthesis, on-the-fly, of gaits with unexpected operational characteristics or even the blending of dynamic manoeuvres lies beyond the capabilities of the current state-of-the-art. In this work we address this limitation by learning a latent space capturing the key stance phas…
The complexity of a legged robot's environment or task can inform how specialised its gait must be to ensure success. Evolving specialised robotic gaits demands many evaluations - acceptable for computer simulations, but not for physical robots. For some tasks, a more general gait, with lower optimization costs, could be satisfactory. In this paper, we introduce a new type of gait controller where complexity can be set by a single parameter, using a dynamic genotype-phenotype mapping. Low controller complexity leads to conservative gaits, while higher complexity allows more sophistication and …
Gait in Eight: Efficient On-Robot Learning for Omnidirectional Quadruped Locomotion
Department of Computer Science, Technical University of Darmstadt（德累斯顿技术大学计算机科学系）；MAB Robtics；German Research Center for AI (DFKI), Research Department: Systems AI for Robot Learning（德国人工智能研究中心（DFKI）系统AI机器人学习研究部）；Robotics Institute Germany (RIG)（德国机器人研究所）
Indian Institute of Technology-Madras（印度理工学院马德拉斯分校）；Indian Institute of Science（印度科学学院）
In this paper, with a view toward fast deployment of learned locomotion gaits in low-cost hardware, we generate a library of walking trajectories, namely, forward trot, backward trot, side-step, and turn in our custom-built quadruped robot, Stoch 2, using reinforcement learning. There are existing approaches that determine optimal policies for each time step, whereas we determine an optimal policy, in the form of end-foot trajectories, for each half walking step i.e., swing phase and stance phase. The way-points for the foot trajectories are obtained from a linear policy, i.e., a linear functi…
Interlimb neural connection is not required for gait transition in quadruped locomotion
Institute of Mathematics for Industry, Kyushu University（工业数学研究所，九州大学）；Research Institute for Electronics Science, Hokkaido University（电子科学研究所，北海道大学）；Research Institute of Electrical Communication, Tohoku University（电气通信研究所，东北大学）；Japan Science and Technology Agency CREST（日本科学技术 Agency CREST）
LoComposition: Terrain-Adaptive Energy-Efficient Quadruped Locomotion without Gait Priors
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；University of Tübingen（图宾根大学）；Technical University of Munich（慕尼黑工业大学）；University of Stuttgart（斯图加特大学）
Walking quadruped robots face challenges in positioning their feet and lifting their legs during gait cycles over uneven terrain. The robot Laika is under development as a quadruped with a flexible, actuated spine designed to assist with foot movement and balance during these gaits. This paper presents the first set of hardware designs for the spine of Laika, a physical prototype of those designs, and tests in both hardware and simulations that show the prototype's capabilities. Laika's spine is a tensegrity structure, used for its advantages with weight and force distribution, and represents …
