Online footstep planning is essential for bipedal walking robots to be able to walk in the presence of disturbances. Until recently this has been achieved by only optimizing the placement of the footstep, keeping the duration of the step constant. In this paper we introduce a footstep planner capable of optimizing footstep placement and timing in real-time by asynchronously combining two optimizers, which we refer to as asynchronous real-time optimization (ARTO). The first optimizer which runs at approximately 25 Hz, utilizes a fourth-order Runge-Kutta (RK4) method to accurately approximate the dynamics of the linear inverted pendulum (LIP) model for bipedal walking, then uses non-linear optimization to find optimal footsteps and duration at a lower frequency. The second optimizer that runs at approximately 250 Hz, uses analytical gradients derived from the full dynamics of the LIP model and constraint penalty terms to perform gradient descent, which finds approximately optimal footstep placement and timing at a higher frequency. By combining the two optimizers asynchronously, ARTO has the benefits of fast reactions to disturbances from the gradient descent optimizer, accurate solutions that avoid local optima from the RK4 optimizer, and increases the probability that a feasible solution will be found from the two optimizers. Experimentally, we show that ARTO is able to recover from considerably larger pushes and produces feasible solutions to larger reference velocity changes than a standard footstep location optimizer, and outperforms using just the RK4 optimizer alone.
Imperial College London（帝国理工学院）；ETH Zürich（苏黎世联邦理工学院）
Online footstep planning is essential for bipedal walking robots, allowing them to walk in the presence of disturbances and sensory noise. Most of the literature on the topic has focused on optimizing the footstep placement while keeping the step timing constant. In this work, we introduce a footstep planner capable of optimizing footstep placement and step time online. The proposed planner, consisting of an Interior Point Optimizer (IPOPT) and an optimizer based on Augmented Lagrangian (AL) method with analytical gradient descent, solves the full dynamics of the Linear Inverted Pendulum (LIP)…
This paper presents an online walking synthesis methodology to enable dynamic and stable walking on constrained footholds for underactuated bipedal robots. Our approach modulates the change of angular momentum about the foot-ground contact pivot at discrete impact using pre-impact vertical center of mass (COM) velocity. To this end, we utilize the underactuated Linear Inverted Pendulum (LIP) model for approximating the underactuated walking dynamics to provide the desired post-impact angular momentum for each step. Desired outputs are constructed via online optimization combined with closed-fo…
A Hybrid passive Linear Inverted Pendulum (HLIP) model is proposed for characterizing, stabilizing and composing periodic orbits for 3D underactuated bipedal walking. Specifically, Period-1 (P1) and Period-2 (P2) orbits are geometrically characterized in the state space of the H-LIP. Stepping controllers are designed for global stabilization of the orbits. Valid ranges of the gains and their optimality are derived. The optimal stepping controller is used to create and stabilize the walking of bipedal robots. An actuated Spring-loaded Inverted Pendulum (aSLIP) model and the underactuated robot …
Collaborative Robotics and Intelligent Systems Institute, Oregon State University（俄勒冈州立大学协作机器人与智能系统研究所）；Agility Robotics（Agility Robotics（敏捷机器人公司））
Recently, work on reinforcement learning (RL) for bipedal robots has successfully learned controllers for a variety of dynamic gaits with robust sim-to-real demonstrations. In order to maintain balance, the learned controllers have full freedom of where to place the feet, resulting in highly robust gaits. In the real world however, the environment will often impose constraints on the feasible footstep locations, typically identified by perception systems. Unfortunately, most demonstrated RL controllers on bipedal robots do not allow for specifying and responding to such constraints. This missi…
This paper systematically decomposes a quadrupedal robot into bipeds to rapidly generate walking gaits and then recomposes these gaits to obtain quadrupedal locomotion. We begin by decomposing the full-order, nonlinear and hybrid dynamics of a three-dimensional quadrupedal robot, including its continuous and discrete dynamics, into two bipedal systems that are subject to external forces. Using the hybrid zero dynamics (HZD) framework, gaits for these bipedal robots can be rapidly generated (on the order of seconds) along with corresponding controllers. The decomposition is achieved in such a w…
This paper presents a novel Adaptive-frequency MPC framework for bipedal locomotion over terrain with uneven stepping stones. In detail, we intend to achieve adaptive foot placement and gait period for bipedal periodic walking gait with this MPC, in order to traverse terrain with discontinuities without slowing down. We pair this adaptive-frequency MPC with a kino-dynamics trajectory optimization for optimal gait periods, center of mass (CoM) trajectory, and foot placements. We use whole-body control (WBC) along with adaptive-frequency MPC to track the optimal trajectories from the offline opt…
We establish stability results for PD tracking control laws in bipedal walking robots. Stability of PD control laws for continuous robotic systems is an established result, and we extend this for hybrid robotic systems, an alternating sequence of continuous and discrete events. Bipedal robots have the leg-swing as the continuous event, and the foot-strike as the discrete event. In addition, bipeds largely have underactuations due to the interactions between feet and ground. For each continuous event, we establish that the convergence rate of the tracking error can be regulated via appropriate …
Extended Hybrid Zero Dynamics for Bipedal Walking of the Knee-less Robot SLIDER
Robot Intelligence Lab, Dyson School of Design Engineering, Imperial College London, UK（机器人智能实验室，设计工程学院，伦敦帝国理工学院，英国）；Department of Aeronautics, Imperial College London, UK（航空系，伦敦帝国理工学院，英国）
We present a framework to generate periodic trajectory references for a 3D under-actuated bipedal robot, using a linear inverted pendulum (LIP) based controller with adaptive neural regulation. We use the LIP template model to estimate the robot's center of mass (CoM) position and velocity at the end of the current step, and formulate a discrete controller that determines the next footstep location to achieve a desired walking profile. This controller is equipped on the frontal plane with a Neural-Network-based adaptive term that reduces the model mismatch between the template and physical rob…
Florida Institute for Human and Machine Cognition（佛罗里达人机认知研究所）
To increase the speed of operation and reduce operator burden, humanoid robots must be able to function autonomously, even in complex, cluttered environments. For this to be possible, they must be able to quickly and efficiently compute desired footsteps to reach a goal. In this work, we present a new A* footstep planner that utilizes a planar region representation of the environment enable footstep planning over rough terrain. To increase the number of available footholds, we present an approach to allow the use of partial footholds during the planning process. The footstep plan solutions are…
National Institute of Advanced Industrial Science and Technology (AIST)（产业技术综合研究所）；University of Tsukuba（筑波大学）
Deep reinforcement learning (RL) based controllers for legged robots have demonstrated impressive robustness for walking in different environments for several robot platforms. To enable the application of RL policies for humanoid robots in real-world settings, it is crucial to build a system that can achieve robust walking in any direction, on 2D and 3D terrains, and be controllable by a user-command. In this paper, we tackle this problem by learning a policy to follow a given step sequence. The policy is trained with the help of a set of procedurally generated step sequences (also called foot…
Global position control for underactuated bipedal walking is a challenging problem due to the lack of actuation on the feet of the robots. In this paper, we apply the Hybrid-Linear Inverted Pendulum (H-LIP) based stepping on 3D underactuated bipedal robots for global position control. The step-to-step (S2S) dynamics of the H-LIP walking approximates the actual S2S dynamics of the walking of the robot, where the step size is considered as the input. Thus the feedback controller based on the H-LIP approximately controls the robot to behave like the H-LIP, the differences between which stay in an…
The ability to track a general walking path with specific timing is crucial to the operational safety and reliability of bipedal robots for avoiding dynamic obstacles, such as pedestrians, in complex environments. This paper introduces an online, full-body motion planner that generates the desired impact-aware motion for fully-actuated bipedal robotic walking. The main novelty of the proposed planner lies in its capability of producing desired motions in real-time that respect the discrete impact dynamics and the desired impact timing. To derive the proposed planner, a full-order hybrid dynami…
Bipedal Robust Walking on Uneven Footholds: Piecewise Slope LIPM with Discrete Model Predictive Control
Faculty of Computing, Harbin Institute of Technology（哈尔滨工业大学计算学部）；State Key Laboratory of Robotics and System, Harbin Institute of Technology（哈尔滨工业大学机器人技术与系统国家重点实验室）
Bipedal robots are a prime example of systems which exhibit highly nonlinear dynamics, underactuation, and undergo complex dissipative impacts. This paper discusses methods used to overcome a wide variety of uncertainties, with the end result being stable bipedal walking. The principal contribution of this paper is to establish sufficiency conditions for yielding input to state stable (ISS) hybrid periodic orbits, i.e., stable walking gaits under model-based and phase-based uncertainties. In particular, it will be shown formally that exponential input to state stabilization (e-ISS) of the cont…
We present a sampling-based kinodynamic planning framework for a bipedal robot in complex environments. Unlike other footstep planner which typically plan footstep locations and the biped dynamics in separate steps, we handle both simultaneously. Three advantages of this approach are (1) the ability to differentiate alternate routes while selecting footstep locations based on the temporal duration of the route as determined by the Linear Inverted Pendulum Model dynamics, (2) the ability to perform collision checking through time so that collisions with moving obstacles are prevented without av…
Dynamic walking on bipedal robots has evolved from an idea in science fiction to a practical reality. This is due to continued progress in three key areas: a mathematical understanding of locomotion, the computational ability to encode this mathematics through optimization, and the hardware capable of realizing this understanding in practice. In this context, this review article outlines the end-to-end process of methods which have proven effective in the literature for achieving dynamic walking on bipedal robots. We begin by introducing mathematical models of locomotion, from reduced order mo…
This paper proposes a novel orientation-aware model predictive control (MPC) for dynamic humanoid walking that can plan footstep locations online. Instead of a point-mass model, this work uses the augmented single rigid body model (aSRBM) to enable the MPC to leverage orientation dynamics and stepping strategy within a unified optimization framework. With the footstep location as part of the decision variables in the aSRBM, the MPC can reason about stepping within the kinematic constraints. A task-space controller (TSC) tracks the body pose and swing leg references output from the MPC, while e…
Developing robust walking controllers for bipedal robots is a challenging endeavor. Traditional model-based locomotion controllers require simplifying assumptions and careful modelling; any small errors can result in unstable control. To address these challenges for bipedal locomotion, we present a model-free reinforcement learning framework for training robust locomotion policies in simulation, which can then be transferred to a real bipedal Cassie robot. To facilitate sim-to-real transfer, domain randomization is used to encourage the policies to learn behaviors that are robust across variat…
Collaborative Robotics and Intelligent Systems Institute, Oregon State University（俄勒冈州立大学协作机器人与智能系统研究所）
In this work, we propose a learning approach for 3D dynamic bipedal walking when footsteps are constrained to stepping stones. While recent work has shown progress on this problem, real-world demonstrations have been limited to relatively simple open-loop, perception-free scenarios. Our main contribution is a more advanced learning approach that enables real-world demonstrations, using the Cassie robot, of closed-loop dynamic walking over moderately difficult stepping-stone patterns. Our approach first uses reinforcement learning (RL) in simulation to train a controller that maps footstep comm…
In this paper, we holistically present a Hybrid-Linear Inverted Pendulum (H-LIP) based approach for synthesizing and stabilizing 3D foot-underactuated bipedal walking, with an emphasis on thorough hardware realization. The H-LIP is proposed to capture the essential components of the underactuated and actuated part of the robotic walking. The robot walking gait is then directly synthesized based on the H-LIP. We comprehensively characterize the periodic orbits of the H-LIP and provably derive the stepping stabilization via its step-to-step (S2S) dynamics, which is then utilized to approximate t…
We present a stepping stabilization control that addresses external push disturbances on bipedal walking robots. The stepping control is synthesized based on the step-to-step (S2S) dynamics of the robot that is controlled to have an approximately constant center of mass (COM) height. We first learn a linear S2S dynamics with bounded model discrepancy from the undisturbed walking behaviors of the robot, where the walking step size is taken as the control input to the S2S dynamics. External pushes are then considered as disturbances to the learned S2S (L-S2S) dynamics. We then apply the system-l…
Stable Walking for Bipedal Locomotion under Foot-Slip via Virtual Nonholonomic Constraints
Centre for Automation and Robotics (CSIC-UPM)（自动化与机器人中心（CSIC-UPM））；Comillas Pontifical University（科米利亚斯主教大学）；IE University（IE大学）；University of Michigan（密歇根大学）
Delft University of Technology（代尔夫特理工大学）；California Institute of Technology（加州理工学院）
Humans are able to negotiate downstep behaviors -- both planned and unplanned -- with remarkable agility and ease. The goal of this paper is to systematically study the translation of this human behavior to bipedal walking robots, even if the morphology is inherently different. Concretely, we begin with human data wherein planned and unplanned downsteps are taken. We analyze this data from the perspective of reduced-order modeling of the human, encoding the center of mass (CoM) kinematics and contact forces, which allows for the translation of these behaviors into the corresponding reduced-ord…
Enhancing Model-Based Step Adaptation for Push Recovery through Reinforcement Learning of Step Timing and Region
Automation and Control Institute, TU Wien（维也纳工业大学自动化与控制研究所）；Autonomous Systems Lab, TU Wien（维也纳工业大学自主系统实验室）；Institute of Robotics and Mechatronics (DLR), German Aerospace Center（德国航空航天中心机器人与机电一体化研究所（DLR））
Machine learning algorithms have found several applications in the field of robotics and control systems. The control systems community has started to show interest towards several machine learning algorithms from the sub-domains such as supervised learning, imitation learning and reinforcement learning to achieve autonomous control and intelligent decision making. Amongst many complex control problems, stable bipedal walking has been the most challenging problem. In this paper, we present an architecture to design and simulate a planar bipedal walking robot(BWR) using a realistic robotics sim…
Florida Institute for Human and Machine Cognition（佛罗里达人机认知研究所）；University of West Florida（西佛罗里达大学）
In trying to build humanoid robots that perform useful tasks in a world built for humans, we address the problem of autonomous locomotion. Humanoid robot planning and control algorithms for walking over rough terrain are becoming increasingly capable. At the same time, commercially available depth cameras have been getting more accurate and GPU computing has become a primary tool in AI research. In this paper, we present a newly constructed behavior control system for achieving fast, autonomous, bipedal walking, without pauses or deliberation. We achieve this using a recently published rapid p…
We present a theoretical analysis of a recent whole body motion planning method, the Randomized Possibility Graph, which uses a high-level decomposition of the feasibility constraint manifold in order to rapidly find routes that may lead to a solution. These routes are then examined by lower-level planners to determine feasibility. In this paper, we show that this approach is probabilistically complete for bipedal robots performing quasi-static walking in "semi-unstructured" environments. Furthermore, we show that the decomposition into higher and lower level planners allows for a considerably…
University of Massachusetts Lowell（马萨诸塞大学洛厄尔分校）；University of Michigan（密歇根大学）；Ohio State University（俄亥俄州立大学）；Purdue University（普渡大学）
Controller design for bipedal walking on dynamic rigid surfaces (DRSes), which are rigid surfaces moving in the inertial frame (e.g., ships and airplanes), remains largely uninvestigated. This paper introduces a hierarchical control approach that achieves stable underactuated bipedal robot walking on a horizontally oscillating DRS. The highest layer of our approach is a real-time motion planner that generates desired global behaviors (i.e., the center of mass trajectories and footstep locations) by stabilizing a reduced-order robot model. One key novelty of this layer is the derivation of the …
Bipedal-Walking-Dynamics Model on Granular Terrains
Department of Mechanical and Aerospace Engineering, Rutgers University（罗格斯大学机械与航空航天工程系）；Bridgewater Raritan High School（布里奇沃特-拉里坦高中）；State Key Lab of Fluid Power Transmission and Control and the School of Mechanical Engineering, Zhejiang University（浙江大学流体动力传动与控制国家重点实验室和机械工程学院）
Real-world applications of bipedal robot walking require accurate, real-time state estimation. State estimation for locomotion over dynamic rigid surfaces (DRS), such as elevators, ships, public transport vehicles, and aircraft, remains under-explored, although state estimator designs for stationary rigid surfaces have been extensively studied. Addressing DRS locomotion in state estimation is a challenging problem mainly due to the nonlinear, hybrid nature of walking dynamics, the nonstationary surface-foot contact points, and hardware imperfections (e.g., limited availability, noise, and drif…
Indian Institute of Technology (BHU) Varanasi（印度理工学院（巴纳拉斯印度教大学）瓦拉纳西分校）；Indian Institute of Technology Roorkee（印度理工学院鲁尔基分校）；Ohio State University（俄亥俄州立大学）；Indian Institute of Science（印度科学学院）
In this paper, with a view toward deployment of light-weight control frameworks for bipedal walking robots, we realize end-foot trajectories that are shaped by a single linear feedback policy. We learn this policy via a model-free and a gradient-free learning algorithm, Augmented Random Search (ARS), in the two robot platforms Rabbit and Digit. Our contributions are two-fold: a) By using torso and support plane orientation as inputs, we achieve robust walking on slopes of up to 20 degrees in simulation. b) We demonstrate additional behaviors like walking backwards, stepping-in-place, and recov…
In this paper, we propose a novel framework on force-and-moment-based Model Predictive Control (MPC) for dynamic legged robots. Specifically, we present a formulation of MPC designed for 10 degree-of-freedom (DoF) bipedal robots using simplified rigid body dynamics with input forces and moments. This MPC controller will calculate the optimal inputs applied to the robot, including 3-D forces and 2-D moments at each foot. These desired inputs will then be generated by mapping these forces and moments to motor torques of 5 actuators on each leg. We evaluate our proposed control design on physical…
Not until recently, robust robot locomotion has been achieved by deep reinforcement learning (DRL). However, for efficient learning of parametrized bipedal walking, developed references are usually required, limiting the performance to that of the references. In this paper, we propose to design an adaptive reward function for imitation learning from the references. The agent is encouraged to mimic the references when its performance is low, while to pursue high performance when it reaches the limit of references. We further demonstrate that developed references can be replaced by low-quality r…
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；New York University（纽约大学）；École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）
In this paper, we propose a novel framework capable of generating various walking and running gaits for bipedal robots. The main goal is to relax the fixed center of mass (CoM) height assumption of the linear inverted pendulum model (LIPM) and generate a wider range of walking and running motions, without a considerable increase in complexity. To do so, we use the concept of virtual constraints in the centroidal space which enables generating motions beyond walking while keeping the complexity at a minimum. By a proper choice of these virtual constraints, we show that we can generate different…
In the control of bipedal locomotion, linear velocity of the center of mass has been widely accepted as a primary variable for summarizing a robot's state vector. The ubiquitous massless-legged linear inverted pendulum (LIP) model is based on it. In this paper, we argue that angular momentum about the contact point has several properties that make it superior to linear velocity for feedback control. So as not to confuse the benefits of angular momentum with any other control design decisions, we first reformulate the standard LIP controller in terms of angular momentum. We then implement the r…
Feedback Regularization and Geometric PID Control for Robust Stabilization of a Planar Three-link Hybrid Bipedal Walking Model
Department of Mechanical Engineering, University of Peradeniya（珀斯德尼亚大学机械工程系）；Postgraduate and Research Unit, Sri Lanka Technological Campus（斯里兰卡科技校园研究生与研究单位）；Department of Mechanical Engineering, Texas Tech University（德克萨斯技术大学机械工程系）
In this paper, previous works on the Model Predictive Control (MPC) and the Divergent Component of Motion (DCM) for bipedal walking control are extended. To this end, we employ a single MPC which uses a combination of Center of Pressure (CoP) manipulation, step adjustment, and Centroidal Moment Pivot (CMP) modulation to design a robust walking controller. Furthermore, we exploit the concept of time-varying DCM to generalize our walking controller for walking in uneven surfaces. Using our scheme, a general and robust walking controller is designed which can be implemented on robots with differe…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
For humanoids to be deployed in demanding situations, such as search and rescue, highly intelligent decision making and proficient sensorimotor skill is expected. A promising solution is to leverage human prowess by interconnecting robot and human via teleoperation. Towards creating seamless operation, this paper presents a dynamic telelocomotion framework that synchronizes the gait of a human pilot with the walking of a bipedal robot. First, we introduce a method to generate a virtual human walking model from the stepping behavior of a human pilot which serves as a reference for the robot to …
The control of bipedal robotic walking remains a challenging problem in the domains of computation and experiment, due to the multi-body dynamics and various sources of uncertainty. In recent years, there has been a rising trend towards model reduction and the design of intuitive controllers to overcome the gap between assumed model and reality. Despite its viability in practical implementation, this local representation of true dynamics naturally indicate limited scalibility towards more dynamical behaviors. With the goal of moving towards increasingly dynamic behaviors, we leverage the detai…
We present a new walking controller based on 3LP, a 3D model of bipedal walking that is composed of three pendulums to simulate falling, swing and torso dynamics. Taking advantage of linear equations and closed-form solutions of 3LP, the proposed controller projects intermediate states of the biped back to the beginning of the phase for which a discrete LQR controller is designed. After the projection, a proper control policy is generated by this LQR controller and used at the intermediate time. The projection controller reacts to disturbances immediately and compared to the discrete LQR contr…
The objective of this work is to design a mechatronic bipedal robot with mobility in 3D environments. The designed robot has a total of six actuated degrees of freedom (DoF), each leg has two DoF located at the hip: one for abduction/adduction and another for thigh flexion/extension, and a third DoF at the knee for the shin flexion/extension. This robot is designed with point-feet legs to achieve a dynamic underactuated walking. Each actuator in the robot includes a DC gear motor, an encoder for position measurement, a flexible joint to form a series flexible actuator, and a feedback controlle…
The Cassie bipedal robot designed by Agility Robotics is providing academics a common platform for sharing and comparing algorithms for locomotion, perception, and navigation. This paper focuses on feedback control for standing and walking using the methods of virtual constraints and gait libraries. The designed controller was implemented six weeks after the robot arrived at the University of Michigan and allowed it to stand in place as well as walk over sidewalks, grass, snow, sand, and burning brush. The controller for standing also enables the robot to ride a Segway. A model of the Cassie r…
A safety-critical measure of legged locomotion performance is a robot's ability to track its desired time-varying position trajectory in an environment, which is herein termed as "global-position tracking". This paper introduces a nonlinear control approach that achieves asymptotic global-position tracking for three-dimensional (3-D) bipedal robot walking. Designing a global-position tracking controller presents a challenging problem due to the complex hybrid robot model and the time-varying desired global-position trajectory. Towards tackling this problem, the first main contribution is the c…
