Model Predictive Control (MPC) is a common tool for the control of nonlinear, real-world systems, such as legged robots. However, solving MPC quickly enough to enable its use in real-time is often challenging. One common solution is given by real-time iterations, which does not solve the MPC problem to convergence, but rather close enough to give an approximate solution. In this paper, we extend this idea to a bilevel control framework where a "high-level" optimization program modifies a controller parameter of a "low-level" MPC problem which generates the control inputs and desired state trajectory. We propose an algorithm to iterate on this bilevel program in real-time and provide conditions for its convergence and improvements in stability. We then demonstrate the efficacy of this algorithm by applying it to a quadrupedal robot where the high-level problem optimizes a contact schedule in real-time. We show through simulation that the algorithm can yield improvements in disturbance rejection and optimality, while creating qualitatively new gaits.
This paper systematically decomposes a quadrupedal robot into bipeds to rapidly generate walking gaits and then recomposes these gaits to obtain quadrupedal locomotion. We begin by decomposing the full-order, nonlinear and hybrid dynamics of a three-dimensional quadrupedal robot, including its continuous and discrete dynamics, into two bipedal systems that are subject to external forces. Using the hybrid zero dynamics (HZD) framework, gaits for these bipedal robots can be rapidly generated (on the order of seconds) along with corresponding controllers. The decomposition is achieved in such a w…
Institute for Infocomm Research, A*STAR（新加坡科技研究局资讯通信研究院）；University College London（伦敦大学学院）
This work developed a learning framework for perceptive legged locomotion that combines visual feedback, proprioceptive information, and active gait regulation of foot-ground contacts. The perception requires only one forward-facing camera to obtain the heightmap, and the active regulation of gait paces and traveling velocity are realized through our formulation of CPG-based high-level imitation of foot-ground contacts. Through this framework, an end-user has the ability to command task-level inputs to control different walking speeds and gait frequencies according to the traversal of differen…
University of Washington（华盛顿大学）；Robotics at Google（Google机器人部门）
We focus on the problem of developing energy efficient controllers for quadrupedal robots. Animals can actively switch gaits at different speeds to lower their energy consumption. In this paper, we devise a hierarchical learning framework, in which distinctive locomotion gaits and natural gait transitions emerge automatically with a simple reward of energy minimization. We use evolutionary strategies (ES) to train a high-level gait policy that specifies gait patterns of each foot, while the low-level convex MPC controller optimizes the motor commands so that the robot can walk at a desired vel…
Beijing Research Institute of UBTECH Robotics（优必选机器人北京研究院）；Tsinghua University（清华大学）
The planning of whole-body motion and step time for bipedal locomotion is constructed as a model predictive control (MPC) problem, in which a sequence of optimization problems needs to be solved online. While directly solving these problems is extremely time-consuming, we propose a predictive gait synthesizer to offer immediate solutions. Based on the full-dimensional model, a library of gaits with different speeds and periods is first constructed offline. Then the proposed gait synthesizer generates real-time gaits at 1kHz by synthesizing the gait library based on the online prediction of cen…
ETH Zürich（苏黎世联邦理工学院）；California Institute of Technology（加州理工学院）
The ability to generate dynamic walking in real-time for bipedal robots with input constraints and underactuation has the potential to enable locomotion in dynamic, complex and unstructured environments. Yet, the high-dimensional nature of bipedal robots has limited the use of full-order rigid body dynamics to gaits which are synthesized offline and then tracked online. In this work we develop an online nonlinear model predictive control approach that leverages the full-order dynamics to realize diverse walking behaviors. Additionally, this approach can be coupled with gaits synthesized offlin…
Simultaneous Contact, Gait and Motion Planning for Robust Multi-Legged Locomotion via Mixed-Integer Convex Optimization
School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA 30332 USA（电气与计算机工程系，佐治亚理工学院，亚特兰大，GA 30332 USA）；Twentieth Century Fox, Springfield, USA（二十世纪福克斯，斯普林菲尔德，USA）；Starfleet Academy, San Francisco, CA 96678 USA（星际舰队学院，旧金山，CA 96678 USA）；Tyrell Inc., 123 Replicant Street, Los Angeles, CA 90210 USA（泰勒尔公司，123 复制人街，洛杉矶，CA 90210 USA）
This work tries to contribute to the design of legged robots with capabilities boosted through thruster-assisted locomotion. Our long-term goal is the development of robots capable of negotiating unstructured environments, including land and air, by leveraging legs and thrusters collaboratively. These robots could be used in a broad number of applications including search and rescue operations, space exploration, automated package handling in residential spaces and digital agriculture, to name a few. In all of these examples, the unique capability of thruster-assisted mobility greatly broadens…
Agile asymmetric multi-legged locomotion: contact planning via geometric mechanics and spin model duality
Department of Mechanical Engineering, Pennsylvania State University（宾夕法尼亚州立大学机械工程系）；MIT Improbable AI Lab, Maschusetts Institute of Technology（麻省理工学院Improbable AI实验室）；Department of Physics, Georgia Institute of Technology（佐治亚理工学院物理系）；Department of Physics, Tsinghua University（清华大学物理系）
The design of gaits for robot locomotion can be a daunting process which requires significant expert knowledge and engineering. This process is even more challenging for robots that do not have an accurate physical model, such as compliant or micro-scale robots. Data-driven gait optimization provides an automated alternative to analytical gait design. In this paper, we propose a novel approach to efficiently learn a wide range of locomotion tasks with walking robots. This approach formalizes locomotion as a contextual policy search task to collect data, and subsequently uses that data to learn…
Learning controllers that reproduce legged locomotion in nature has been a long-time goal in robotics and computer graphics. While yielding promising results, recent approaches are not yet flexible enough to be applicable to legged systems of different morphologies. This is partly because they often rely on precise motion capture references or elaborate learning environments that ensure the naturality of the emergent locomotion gaits but prevent generalization. This work proposes a generic approach for ensuring realism in locomotion by guiding the learning process with the spring-loaded invert…
Gait-Adaptive Perceptive Humanoid Locomotion with Real-Time Under-Base Terrain Reconstruction
Department of Electronic Engineering and Information Science (EEIS), University of Science and Technology of China（电子工程与信息科学系，中国科学技术大学）；LimX Dynamics（LimX动力学）；Hong Kong University of Science and Technology（香港科技大学）；School of Mechanics Engineering, Harbin Institute of Technology (HIT)（机械工程学院，哈尔滨工业大学）；Department of Computing, The Hong Kong Polytechnic University（计算学院，香港理工大学）；Zhejiang University-University of Illinois Urbana-Champaign Institute (ZJUI)（浙江大学-伊利诺伊大学厄巴纳-香槟分校联合研究所）
Non-Gaited Legged Locomotion with Monte-Carlo Tree Search and Supervised Learning
Dynamic Legged Systems Laboratory, Istituto Italiano di Tecnologia (IIT), Genova, Italy（动态腿系统实验室，意大利技术研究院（IIT），意大利热那亚）；D AI Laboratory, Technical University of Munich (TUM), Germany（3D人工智能实验室，慕尼黑技术大学（TUM），德国）；ATARI Laboratory, MIRMI, Technical University of Munich (TUM), Germany（ATARI实验室，MIRMI，慕尼黑技术大学（TUM），德国）
Real-time constraint satisfaction for robots can be quite challenging due to the high computational complexity that arises when accounting for the system dynamics and environmental interactions, often requiring simplification in modelling that might not necessarily account for all performance criteria. We instead propose an optimization-free approach where reference trajectories are manipulated to satisfy constraints brought on by ground contact as well as those prescribed for states and inputs. Unintended changes to trajectories especially ones optimized to produce periodic gaits can adversel…
Convex model predictive controls (MPCs) with a single rigid body model have demonstrated strong performance on real legged robots. However, convex MPCs are limited by their assumptions such as small rotation angle and pre-defined gait, limiting the richness of potential solutions. We remove those assumptions and solve the complete mixed-integer non-convex programming with single rigid body model. We first collect datasets of pre-solved problems offline, then learn the problem-solution map to solve this optimization fast for MPC. If warm-starts can be found, offline problems can be solved close…
It is often overlooked by roboticists when designing locomotion controllers for their legged machines, that energy consumption plays an important role in selecting the best gaits for locomotion at high speeds or over long distances. The purpose of this study is to examine four similar asymmetrical quadrupedal gaits that are frequently observed in legged animals in nature. To understand how a specific footfall pattern will change the energetics of a legged system, we first developed a full body model of a quadrupedal robot called A1. And for each gait we created a hybrid system with desired foo…
University of California, Berkeley（加州大学伯克利分校）；University of California, Santa Cruz（加州大学圣克鲁兹分校）；Autodesk, Inc.（欧特克公司）；Nasa Ames Research Center（NASA艾姆斯研究中心）；OpenAI；International Computer Science Institute（国际计算机科学研究所）
Tensegrity robots, composed of rigid rods connected by elastic cables, have a number of unique properties that make them appealing for use as planetary exploration rovers. However, control of tensegrity robots remains a difficult problem due to their unusual structures and complex dynamics. In this work, we show how locomotion gaits can be learned automatically using a novel extension of mirror descent guided policy search (MDGPS) applied to periodic locomotion movements, and we demonstrate the effectiveness of our approach on tensegrity robot locomotion. We evaluate our method with real-world…
Many robotics applications, from object manipulation to locomotion, require planning methods that are capable of handling the dynamics of contact. Trajectory optimization has been shown to be a viable approach that can be made to support contact dynamics. However, the current state-of-the art methods remain slow and are often difficult to get to converge. In this work, we leverage recent advances in bilevel optimization to design an algorithm capable of efficiently generating trajectories that involve making and breaking contact. We demonstrate our method's efficiency by outperforming an alter…
Both legged locomotion and slithering motions typically utilize periodic gaits -- repeating cycles of body shape change that produce a net motion through the world. Legged locomotion can be viewed from the perspective of piece-wise contact constraint formation and removal. Slithering and low Reynolds number swimming operate under continuous constraints of force balance, wherein dissipation removes the ability to accumulate momentum. Here we discuss how to bridge the gap between these domains of motion, thereby, among other benefits, producing models for the space of legged locomotion with slip…
Planning locomotion trajectories for legged microrobots is challenging because of their complex morphology, high frequency passive dynamics, and discontinuous contact interactions with their environment. Consequently, such research is often driven by time-consuming experimental methods. As an alternative, we present a framework for systematically modeling, planning, and controlling legged microrobots. We develop a three-dimensional dynamic model of a 1.5 gram quadrupedal microrobot with complexity (e.g., number of degrees of freedom) similar to larger-scale legged robots. We then adapt a recen…
The Geometry of Optimal Gait Families for Steering Kinematic Locomoting Systems
Collaborative Robotics and Intelligent Systems (CoRIS) Institute at Oregon State University（俄勒冈州立大学协作机器人与智能系统（CoRIS）研究所）；Johns Hopkins University（约翰斯·霍普金斯大学）
University of Edinburgh（爱丁堡大学）；University College London（伦敦大学学院）
This work developed a kernel-based residual learning framework for quadrupedal robotic locomotion. Initially, a kernel neural network is trained with data collected from an MPC controller. Alongside a frozen kernel network, a residual controller network is trained via reinforcement learning to acquire generalized locomotion skills and resilience against external perturbations. With this proposed framework, a robust quadrupedal locomotion controller is learned with high sample efficiency and controllability, providing omnidirectional locomotion at continuous velocities. Its versatility and robu…
Robotic Systems Lab, ETH Zurich（苏黎世联邦理工学院机器人系统实验室）；Robotics and Artificial Intelligence Lab, KAIST（韩国科学技术院机器人与人工智能实验室）；Intelligent Systems Lab, Intel（英特尔智能系统实验室）
Legged robots that can operate autonomously in remote and hazardous environments will greatly increase opportunities for exploration into under-explored areas. Exteroceptive perception is crucial for fast and energy-efficient locomotion: perceiving the terrain before making contact with it enables planning and adaptation of the gait ahead of time to maintain speed and stability. However, utilizing exteroceptive perception robustly for locomotion has remained a grand challenge in robotics. Snow, vegetation, and water visually appear as obstacles on which the robot cannot step~-- or are missing …
The robotic locomotion community is interested in optimal gaits for control. Based on the optimization criterion, however, there could be a number of possible optimal gaits. For example, the optimal gait for maximizing displacement with respect to cost is quite different from the maximum displacement optimal gait. Beyond these two general optimal gaits, we believe that the optimal gait should deal with various situations for high-resolution of motion planning, e.g., steering the robot or moving in "baby steps." As the step size or steering ratio increases or decreases, the optimal gaits will s…
In this paper, we propose a method for generating undulatory gaits for snake robots. Instead of starting from a pre-defined movement pattern such as a serpenoid curve, we use a Model Predictive Control approach to automatically generate effective locomotion gaits via trajectory optimization. An important advantage of this approach is that the resulting gaits are automatically adapted to the environment that is being modeled as part of the snake dynamics. To illustrate this, we use a novel model for anisotropic dry friction, along with existing models for viscous friction and fluid dynamic effe…
Korea Advanced Institute of Science and Technology（韩国科学技术院）；Istituto Italiano di Tecnologia（意大利理工学院）
In this work, a non-gaited framework for legged system locomotion is presented. The approach decouples the gait sequence optimization by considering the problem as a decision-making process. The redefined contact sequence problem is solved by utilizing a Monte Carlo Tree Search (MCTS) algorithm that exploits optimization-based simulations to evaluate the best search direction. The proposed scheme has proven to have a good trade-off between exploration and exploitation of the search space compared to the state-of-the-art Mixed-Integer Quadratic Programming (MIQP). The model predictive control (…
To improve the control of wearable robotics for gait assistance, we present an approach for continuous locomotion mode recognition as well as gait phase and stair slope estimation based on artificial neural networks that include time history information. The input features consist exclusively of processed variables that can be measured with a single shank-mounted inertial measurement unit. We introduce a wearable device to acquire real-world environment test data to demonstrate the performance and the robustness of the approach. Mean absolute error (gait phase, stair slope) and accuracy (locom…
The common approach for local navigation on challenging environments with legged robots requires path planning, path following and locomotion, which usually requires a locomotion control policy that accurately tracks a commanded velocity. However, by breaking down the navigation problem into these sub-tasks, we limit the robot's capabilities since the individual tasks do not consider the full solution space. In this work, we propose to solve the complete problem by training an end-to-end policy with deep reinforcement learning. Instead of continuously tracking a precomputed path, the robot nee…
California Institute of Technology（加州理工学院）；Northeastern University（东北大学）
Flying animals, such as bats, fly through their fluidic environment as they create air jets and form wake structures downstream of their flight path. Bats, in particular, dynamically morph their highly flexible and dexterous armwing to manipulate their fluidic environment which is key to their agility and flight efficiency. This paper presents the theoretical and numerical analysis of the wake-structure-based gait design inspired by bat flight for flapping robots using the notion of reduced-order models and unsteady aerodynamic model incorporating Wagner function. The objective of this paper i…
This paper presents a Q-learning framework for learning optimal locomotion gaits in robotic systems modeled as coupled rigid bodies. Inspired by prevalence of periodic gaits in bio-locomotion, an open loop periodic input is assumed to (say) affect a nominal gait. The learning problem is to learn a new (modified) gait by using only partial noisy measurements of the state. The objective of learning is to maximize a given reward modeled as an objective function in optimal control settings. The proposed control architecture has three main components: (i) Phase modeling of dynamics by a single phas…
