The main drawbacks of input-output linearizing controllers are the need for precise dynamics models and not being able to account for input constraints. Model uncertainty is common in almost every robotic application and input saturation is present in every real world system. In this paper, we address both challenges for the specific case of bipedal robot control by the use of reinforcement learning techniques. Taking the structure of a standard input-output linearizing controller, we use an additive learned term that compensates for model uncertainty. Moreover, by adding constraints to the learning problem we manage to boost the performance of the final controller when input limits are present. We demonstrate the effectiveness of the designed framework for different levels of uncertainty on the five-link planar walking robot RABBIT.
Developing robust walking controllers for bipedal robots is a challenging endeavor. Traditional model-based locomotion controllers require simplifying assumptions and careful modelling; any small errors can result in unstable control. To address these challenges for bipedal locomotion, we present a model-free reinforcement learning framework for training robust locomotion policies in simulation, which can then be transferred to a real bipedal Cassie robot. To facilitate sim-to-real transfer, domain randomization is used to encourage the policies to learn behaviors that are robust across variat…
University of California Berkeley（加州大学伯克利分校）；Carnegie Mellon University（卡内基梅隆大学）
Recent advances in legged locomotion have enabled quadrupeds to walk on challenging terrains. However, bipedal robots are inherently more unstable and hence it's harder to design walking controllers for them. In this work, we leverage recent advances in rapid adaptation for locomotion control, and extend them to work on bipedal robots. Similar to existing works, we start with a base policy which produces actions while taking as input an estimated extrinsics vector from an adaptation module. This extrinsics vector contains information about the environment and enables the walking controller to …
We present a novel approach to control design for nonlinear systems which leverages model-free policy optimization techniques to learn a linearizing controller for a physical plant with unknown dynamics. Feedback linearization is a technique from nonlinear control which renders the input-output dynamics of a nonlinear plant \emph{linear} under application of an appropriate feedback controller. Once a linearizing controller has been constructed, desired output trajectories for the nonlinear plant can be tracked using a variety of linear control techniques. However, the calculation of a lineariz…
Through the method of Learning Feedback Linearization, we seek to learn a linearizing controller to simplify the process of controlling a car to race autonomously. A soft actor-critic approach is used to learn a decoupling matrix and drift vector that effectively correct for errors in a hand-designed linearizing controller. The result is an exactly linearizing controller that can be used to enable the well-developed theory of linear systems to design path planning and tracking schemes that are easy to implement and significantly less computationally demanding. To demonstrate the method of feed…
Learning Bipedal Walking for Humanoids with Current Feedback
CNRS-AIST JRL (Joint Robotics Laboratory) IRL（法国国家科学研究中心-产业技术综合研究所联合机器人实验室国际联合实验室）；National Institute of Advanced Industrial Science and Technology (AIST)（产业技术综合研究所）；University of Tsukuba（筑波大学）；Stanford University（斯坦福大学）
Teaching an anthropomorphic robot from human example offers the opportunity to impart humanlike qualities on its movement. In this work we present a reinforcement learning based method for teaching a real world bipedal robot to perform movements directly from human motion capture data. Our method seamlessly transitions from training in a simulation environment to executing on a physical robot without requiring any real world training iterations or offline steps. To overcome the disparity in joint configurations between the robot and the motion capture actor, our method incorporates motion re-t…
EmoBipedNav: Emotion-aware Social Navigation for Bipedal Robots with Deep Reinforcement Learning
Laboratory for Intelligent Decision and Autonomous Robots, Woodruff School of Mechanical Engineering, Georgia Institute of Technology（智能决策与自主机器人实验室，伍德鲁夫机械工程学院，佐治亚理工学院）；College of Engineering and Petroleum, Kuwait University（工程与石油学院，科威特大学）；School of Computational Science and Engineering, Georgia Institute of Technology（计算科学与工程学院，佐治亚理工学院）；Khoury College of Computer Sciences, Northeastern University（计算机科学学院，东北大学）
In the control of bipedal locomotion, linear velocity of the center of mass has been widely accepted as a primary variable for summarizing a robot's state vector. The ubiquitous massless-legged linear inverted pendulum (LIP) model is based on it. In this paper, we argue that angular momentum about the contact point has several properties that make it superior to linear velocity for feedback control. So as not to confuse the benefits of angular momentum with any other control design decisions, we first reformulate the standard LIP controller in terms of angular momentum. We then implement the r…
National Institute of Advanced Industrial Science and Technology (AIST)（产业技术综合研究所）；University of Tsukuba（筑波大学）
Deep reinforcement learning (RL) based controllers for legged robots have demonstrated impressive robustness for walking in different environments for several robot platforms. To enable the application of RL policies for humanoid robots in real-world settings, it is crucial to build a system that can achieve robust walking in any direction, on 2D and 3D terrains, and be controllable by a user-command. In this paper, we tackle this problem by learning a policy to follow a given step sequence. The policy is trained with the help of a set of procedurally generated step sequences (also called foot…
This paper presents a gait controller for bipedal robots to achieve highly agile walking over various terrains given local slope and friction cone information. Without these considerations, untimely impacts can cause a robot to trip and inadequate tangential reaction forces at the stance foot can cause slippages. We address these challenges by combining, in a novel manner, a model based on an Angular Momentum Linear Inverted Pendulum (ALIP) and a Model Predictive Control (MPC) foot placement planner that is executed by the method of virtual constraints. The process starts with abstracting from…
A Hybrid passive Linear Inverted Pendulum (HLIP) model is proposed for characterizing, stabilizing and composing periodic orbits for 3D underactuated bipedal walking. Specifically, Period-1 (P1) and Period-2 (P2) orbits are geometrically characterized in the state space of the H-LIP. Stepping controllers are designed for global stabilization of the orbits. Valid ranges of the gains and their optimality are derived. The optimal stepping controller is used to create and stabilize the walking of bipedal robots. An actuated Spring-loaded Inverted Pendulum (aSLIP) model and the underactuated robot …
We present a framework to generate periodic trajectory references for a 3D under-actuated bipedal robot, using a linear inverted pendulum (LIP) based controller with adaptive neural regulation. We use the LIP template model to estimate the robot's center of mass (CoM) position and velocity at the end of the current step, and formulate a discrete controller that determines the next footstep location to achieve a desired walking profile. This controller is equipped on the frontal plane with a Neural-Network-based adaptive term that reduces the model mismatch between the template and physical rob…
This paper systematically decomposes a quadrupedal robot into bipeds to rapidly generate walking gaits and then recomposes these gaits to obtain quadrupedal locomotion. We begin by decomposing the full-order, nonlinear and hybrid dynamics of a three-dimensional quadrupedal robot, including its continuous and discrete dynamics, into two bipedal systems that are subject to external forces. Using the hybrid zero dynamics (HZD) framework, gaits for these bipedal robots can be rapidly generated (on the order of seconds) along with corresponding controllers. The decomposition is achieved in such a w…
In this paper, we propose a multi-domain control parameter learning framework that combines Bayesian Optimization (BO) and Hybrid Zero Dynamics (HZD) for locomotion control of bipedal robots. We leverage BO to learn the control parameters used in the HZD-based controller. The learning process is firstly deployed in simulation to optimize different control parameters for a large repertoire of gaits. Next, to tackle the discrepancy between the simulation and the real world, the learning process is applied on the physical robot to learn for corrections to the control parameters learned in simulat…
Northeastern University（东北大学）；University of Alabama（阿拉巴马大学）
We will report our efforts in designing feedback for the thruster-assisted walking of a bipedal robot. We will assume for well-tuned supervisory controllers and will focus on fine-tuning the desired joint trajectories to satisfy the performance being sought. In doing this, we will devise an intermediary filter based on the emerging idea of reference governors. Since these modifications and impact events lead to deviations from the desired periodic orbits, we will guarantee hybrid invariance in a robust fashion by applying predictive schemes within a short time envelope during the double suppor…
Global position control for underactuated bipedal walking is a challenging problem due to the lack of actuation on the feet of the robots. In this paper, we apply the Hybrid-Linear Inverted Pendulum (H-LIP) based stepping on 3D underactuated bipedal robots for global position control. The step-to-step (S2S) dynamics of the H-LIP walking approximates the actual S2S dynamics of the walking of the robot, where the step size is considered as the input. Thus the feedback controller based on the H-LIP approximately controls the robot to behave like the H-LIP, the differences between which stay in an…
In this paper, we propose a novel framework on force-and-moment-based Model Predictive Control (MPC) for dynamic legged robots. Specifically, we present a formulation of MPC designed for 10 degree-of-freedom (DoF) bipedal robots using simplified rigid body dynamics with input forces and moments. This MPC controller will calculate the optimal inputs applied to the robot, including 3-D forces and 2-D moments at each foot. These desired inputs will then be generated by mapping these forces and moments to motor torques of 5 actuators on each leg. We evaluate our proposed control design on physical…
The objective of this work is to design a mechatronic bipedal robot with mobility in 3D environments. The designed robot has a total of six actuated degrees of freedom (DoF), each leg has two DoF located at the hip: one for abduction/adduction and another for thigh flexion/extension, and a third DoF at the knee for the shin flexion/extension. This robot is designed with point-feet legs to achieve a dynamic underactuated walking. Each actuator in the robot includes a DC gear motor, an encoder for position measurement, a flexible joint to form a series flexible actuator, and a feedback controlle…
Evaluating Robots Like Human Infants: A Case Study of Learned Bipedal Locomotion
Department of Electrical Engineering and Computer Sciences, Oregon State University（电子工程与计算机科学系，俄勒冈州立大学）；Department of Psychology, New York University（心理学系，纽约大学）
Feedback Regularization and Geometric PID Control for Robust Stabilization of a Planar Three-link Hybrid Bipedal Walking Model
Department of Mechanical Engineering, University of Peradeniya（珀斯德尼亚大学机械工程系）；Postgraduate and Research Unit, Sri Lanka Technological Campus（斯里兰卡科技校园研究生与研究单位）；Department of Mechanical Engineering, Texas Tech University（德克萨斯技术大学机械工程系）
Low-dimensional models are ubiquitous in the bipedal robotics literature. On the one hand is the community of researchers that bases feedback control design on pendulum models selected to capture the center of mass dynamics of the robot during walking. On the other hand is the community that bases feedback control design on virtual constraints, which induce an exact low-dimensional model in the closed-loop system. In the first case, the low-dimensional model is valued for its physical insight and analytical tractability. In the second case, the low-dimensional model is integral to a rigorous a…
In this paper, a novel optimal technique for joint angles trajectory tracking control with energy optimization for a biped robot with toe foot is proposed. For the task of climbing stairs by a 9-link biped model, a cycloid trajectory for swing phase is proposed in such a way that the cycloid variables depend on the staircase dimensions. Zero Moment Point(ZMP) criteria is taken for satisfying stability constraint. This paper mainly can be divided into 3 steps: 1) Planning stable cycloid trajectory for initial step and subsequent step for climbing upstairs and Inverse Kinematics using an unsuper…
Despite many accomplishments by legged robot designers, state-of-the-art bipedal robots are prone to falling over, cannot negotiate extremely rough terrains and cannot directly regulate unilateral contact forces. Our objective is to integrate merits of legged and aerial robots in a single platform. We will show that the thrusters in a bipedal legged robot called Harpy can be leveraged to stabilize the robot's frontal dynamics and permit jumping over large obstacles which is an unusual capability not reported before. In addition, we will capitalize on the thrusters action in Harpy and will show…
This paper proposes a feedback linearizing law for single-track dynamic models, allowing the design of a trajectory tracking controller exploiting linear control theory. The main characteristics of this algorithm are its simplicity, its independence from any vehicle model parameter, apart from the position of the center of mass, and its robustness. In particular, a numerical bifurcation analysis demonstrates that, for physically meaningful values of the center of mass deviation, the equilibrium is structurally asymptotically stable. Experimental results, concerning the linearising law and its …
