Biped robots have plenty of benefits over wheeled, quadruped, or hexapod robots due to their ability to behave like human beings in tough and non-flat environments. Deformable terrain is another challenge for biped robots as it has to deal with sinkage and maintain stability without falling. In this study, we are proposing a Deep Deterministic Policy Gradient (DDPG) approach for motion control of a flat-foot biped robot walking on deformable terrain. We have considered a 7-link biped robot for our proposed approach. For soft soil terrain modeling, we have considered triangular Mesh to describe its geometry, where mesh parameters determine the softness of soil. All simulations have been performed on PyChrono, which can handle soft soil environments.
The ability to generate robust walking gaits on bipedal robots is key to their successful realization on hardware. To this end, this work extends the method of Hybrid Zero Dynamics (HZD) -- which traditionally only accounts for locomotive stability via periodicity constraints under perfect impact events -- through the inclusion of the saltation matrix with a view toward synthesizing robust walking gaits. By jointly minimizing the norm of the extended saltation matrix and the torque of the robot directly in the gait generation process, we demonstrate that the synthesized gaits are more robust t…
This paper describes a topological approach to generating families of open- and closed-loop walking gaits for underactuated 2D and 3D biped walkers subject to configuration inequality constraints, physical holonomic constraints (e.g.,closed-loop linkages), and virtual holonomic constraints (user-defined constraints enforced through feedback control). Our method constructs implicitly-defined manifolds of feasible periodic gaits within a state-time-control space that parameterizes the biped's hybrid trajectories. Since equilibrium configurations of the biped often belong to such manifolds, we us…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Laban/Bartenieff Institute of Movement Studies（拉班/芭特妮芙动作研究学院）
Humans are efficient, yet expressive in their motion. Human walking behaviors can be used to walk across a great variety of surfaces without falling and to communicate internal state to other humans through variable gait styles. This provides inspiration for creating similarly expressive bipedal robots. To this end, a framework is presented for stylistic gait generation in a compass-like under-actuated planar biped model. The gait design is done using model-based trajectory optimization with variable constraints. For a finite range of optimization parameters, a large set of 360 gaits can be ge…
Beijing Research Institute of UBTECH Robotics（优必选机器人北京研究院）；Tsinghua University（清华大学）
The planning of whole-body motion and step time for bipedal locomotion is constructed as a model predictive control (MPC) problem, in which a sequence of optimization problems needs to be solved online. While directly solving these problems is extremely time-consuming, we propose a predictive gait synthesizer to offer immediate solutions. Based on the full-dimensional model, a library of gaits with different speeds and periods is first constructed offline. Then the proposed gait synthesizer generates real-time gaits at 1kHz by synthesizing the gait library based on the online prediction of cen…
National Institute of Applied Sciences and Technology（国立应用科学与技术学院）
In this paper, a simple trajectory generation method for biped walking is proposed. The dynamic model of the five link bipedal robot is first reduced using several biologically inspired assumptions. A sinusoidal curve is then imposed to the ankle of the swing leg's trajectory. The reduced model is finally obtained and solved: it is an homogeneous second order differential equations with constant coefficients. The algebraic solution obtained ensures a stable rhythmic gait for the bipedal robot. It's continuous in the defined time interval, easy to implement when the boundary conditions are well…
ETH Zürich（苏黎世联邦理工学院）；California Institute of Technology（加州理工学院）
The ability to generate dynamic walking in real-time for bipedal robots with input constraints and underactuation has the potential to enable locomotion in dynamic, complex and unstructured environments. Yet, the high-dimensional nature of bipedal robots has limited the use of full-order rigid body dynamics to gaits which are synthesized offline and then tracked online. In this work we develop an online nonlinear model predictive control approach that leverages the full-order dynamics to realize diverse walking behaviors. Additionally, this approach can be coupled with gaits synthesized offlin…
University of Oslo（奥斯陆大学）；Norwegian Defence Research Institute（挪威国防研究所）
Legged robots promise higher versatility and the ability to traverse much more difficult terrains than their wheeled counterparts. Even though the use of legged robots have increased drastically in the last few years, they are still not close to the performance seen from legged animals in nature. Robotic legs are typically fairly simple mechanically, and few feature an ankle joint, even though most land mammals have one. The ankle could be a key to better performance and stability for legged robots, and in this paper we investigate how the use of an ankle in a bipedal robot could improve its p…
National School of Engineering of Sfax, University of Sfax（斯法克斯国家工程学院，斯法克斯大学）
In this chapter we will highlight our experimental studies on natural human walking analysis and introduce a biologically inspired design for simple bipedal locomotion system of humanoid robots. Inspiration comes directly from human walking analysis and human muscles mechanism and control. A hybrid algorithm for walking gaits generation is then proposed as an innovative alternative to classically used kinematics and dynamic equations solving, the gaits include knee, ankle and hip trajectories. The proposed algorithm is an intelligent evolutionary based on particle swarm optimization paradigm. …
This work presents Neural Gaits, a method for learning dynamic walking gaits through the enforcement of set invariance that can be refined episodically using experimental data from the robot. We frame walking as a set invariance problem enforceable via control barrier functions (CBFs) defined on the reduced-order dynamics quantifying the underactuated component of the robot: the zero dynamics. Our approach contains two learning modules: one for learning a policy that satisfies the CBF condition, and another for learning a residual dynamics model to refine imperfections of the nominal model. Im…
Walking controllers often require parametrization which must be tuned according to some cost function. To estimate these parameters, simulations can be performed which are cheap but do not fully represent reality. Real-robot experiments, on the other hand, are more expensive and lead to hardware wear-off. In this paper, we propose an approach for combining simulations and real experiments to learn gait stabilization parameters. We use a Bayesian optimization method which selects the most informative points in parameter space to evaluate based on the entropy of the cost function to optimize. Ex…
University of São Paulo（圣保罗大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
In bipedal gait design literature, one of the common ways of generating stable 3D walking gait is by designing the frontal and sagittal controllers as decoupled dynamics. The study of the decoupled frontal dynamics is, however, still understudied if compared with the sagittal dynamics. In this paper it is presented a formal approach to the problem of frontal dynamics stabilization by extending the hybrid zero dynamics framework to deal with the frontal gait design problem.
Laboratoire des Sciences du Numérique de Nantes (LS2N)（南特数字科学实验室）；CNRS（法国国家科学研究中心）；Ecole Centrale de Nantes（南特中央理工学院）；Université de Nantes（南特大学）；École Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院）；University of Michigan（密歇根大学）
This paper seeks insight into stabilization mechanisms for periodic walking gaits in 3D bipedal robots. Based on this insight, a control strategy based on virtual constraints, which imposes coordination between joints rather than a temporal evolution, will be proposed for achieving asymptotic convergence toward a periodic motion. For planar bipeds with one degree of underactuation, it is known that a vertical displacement of the center of mass---with downward velocity at the step transition---induces stability of a walking gait. This paper concerns the qualitative extension of this type of pro…
In this paper, we holistically present a Hybrid-Linear Inverted Pendulum (H-LIP) based approach for synthesizing and stabilizing 3D foot-underactuated bipedal walking, with an emphasis on thorough hardware realization. The H-LIP is proposed to capture the essential components of the underactuated and actuated part of the robotic walking. The robot walking gait is then directly synthesized based on the H-LIP. We comprehensively characterize the periodic orbits of the H-LIP and provably derive the stepping stabilization via its step-to-step (S2S) dynamics, which is then utilized to approximate t…
California Institute of Technology（加州理工学院）；Technion - Israel Institute of Technology（以色列理工学院）
Dynamic bipedal robot locomotion has achieved remarkable success due in part to recent advances in trajectory generation and nonlinear control for stabilization. A key assumption utilized in both theory and experiments is that the robot's stance foot always makes no-slip contact with the ground, including at impacts. This assumption breaks down on slippery low-friction surfaces, as commonly encountered in outdoor terrains, leading to failure and loss of stability. In this work, we extend the theoretical analysis and trajectory optimization to account for stick-slip transitions at point foot co…
Bipedal walking is one of the most difficult but exciting challenges in robotics. The difficulties arise from the complexity of high-dimensional dynamics, sensing and actuation limitations combined with real-time and computational constraints. Deep Reinforcement Learning (DRL) holds the promise to address these issues by fully exploiting the robot dynamics with minimal craftsmanship. In this paper, we propose a novel DRL approach that enables an agent to learn omnidirectional locomotion for humanoid (bipedal) robots. Notably, the locomotion behaviors are accomplished by a single control policy…
Tsinghua University（清华大学）；Technical University of Munich（慕尼黑工业大学）
Not until recently, robust bipedal locomotion has been achieved through reinforcement learning. However, existing implementations rely heavily on insights and efforts from human experts, which is costly for the iterative design of robot systems. Also, styles of the learned motion are strictly limited to that of the reference. In this paper, we propose a new way to learn bipedal locomotion from a simple sine wave as the reference for foot heights. With the naive human insight that the two feet should be lifted up alternatively and periodically, we experimentally demonstrate on the Cassie robot …
This paper systematically decomposes a quadrupedal robot into bipeds to rapidly generate walking gaits and then recomposes these gaits to obtain quadrupedal locomotion. We begin by decomposing the full-order, nonlinear and hybrid dynamics of a three-dimensional quadrupedal robot, including its continuous and discrete dynamics, into two bipedal systems that are subject to external forces. Using the hybrid zero dynamics (HZD) framework, gaits for these bipedal robots can be rapidly generated (on the order of seconds) along with corresponding controllers. The decomposition is achieved in such a w…
Collaborative Robotics and Intelligent Systems Institute, Oregon State University（俄勒冈州立大学协作机器人与智能系统研究所）；Agility Robotics（Agility Robotics（敏捷机器人公司））
Recently, work on reinforcement learning (RL) for bipedal robots has successfully learned controllers for a variety of dynamic gaits with robust sim-to-real demonstrations. In order to maintain balance, the learned controllers have full freedom of where to place the feet, resulting in highly robust gaits. In the real world however, the environment will often impose constraints on the feasible footstep locations, typically identified by perception systems. Unfortunately, most demonstrated RL controllers on bipedal robots do not allow for specifying and responding to such constraints. This missi…
This paper describes a new approach allowing the generation of a simplified Biped gait. This approach combines a classical dynamic modeling with an inverse kinematics' solver based on particle swarm optimization, PSO. First, an inverted pendulum, IP, is used to obtain a simplified dynamic model of the robot and to compute the target position of a key point in biped locomotion, the Centre Of Mass, COM. The proposed algorithm, called IK-PSO, Inverse Kinematics PSO, returns and inverse kinematics solution corresponding to that COM respecting the joints constraints. In This paper the inertia weigh…
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；Technische Universität Ilmenau（伊尔默瑙工业大学）
The hybrid zero dynamics control concept for bipedal walking is extended to include a non-instantaneous double support phase. A symmetric robot that consists of five rigid body segments which are connected by four actuated revolute joints is considered. Periodic walking gaits with a constant average walking speed consists of alternating single (SSP) and double support phases (DSP). Hybrid zero dynamics control designs usually assume an instantaneous DSP, which is a severe limitation. The proposed controllers use continuous SSPs and DSPs. Transitions between both phases are modeled as instantan…
Southern University of Science and Technology（南方科技大学）；The University of British Columbia（不列颠哥伦比亚大学）；National University of Singapore（新加坡国立大学）
Controlling a biped robot to walk stably is a challenging task considering its nonlinearity and hybrid dynamics. Reinforcement learning can address these issues by directly mapping the observed states to optimal actions that maximize the cumulative reward. However, the local minima caused by unsuitable rewards and the overestimation of the cumulative reward impede the maximization of the cumulative reward. To increase the cumulative reward, this paper designs a gait reward based on walking principles, which compensates the local minima for unnatural motions. Besides, an Adversarial Twin Delaye…
This paper presents a framework that leverages both control theory and machine learning to obtain stable and robust bipedal locomotion without the need for manual parameter tuning. Traditionally, gaits are generated through trajectory optimization methods and then realized experimentally -- a process that often requires extensive tuning due to differences between the models and hardware. In this work, the process of gait realization via hybrid zero dynamics (HZD) based optimization is formally combined with preference-based learning to systematically realize dynamically stable walking. Importa…
In this research article, we have reported periodic cellular automata rules for different gait state prediction and classification of the gait data using extreme machine Leaning (ELM). This research is the first attempt to use cellular automaton to understand the complexity of bipedal walk. Due to nonlinearity, varying configurations throughout the gait cycle and the passive joint located at the unilateral foot-ground contact in bipedal walk resulting variation of dynamic descriptions and control laws from phase to phase for human gait is making difficult to predict the bipedal walk states. We…
This paper presents a novel Adaptive-frequency MPC framework for bipedal locomotion over terrain with uneven stepping stones. In detail, we intend to achieve adaptive foot placement and gait period for bipedal periodic walking gait with this MPC, in order to traverse terrain with discontinuities without slowing down. We pair this adaptive-frequency MPC with a kino-dynamics trajectory optimization for optimal gait periods, center of mass (CoM) trajectory, and foot placements. We use whole-body control (WBC) along with adaptive-frequency MPC to track the optimal trajectories from the offline opt…
CNRS（法国国家科学研究中心）；IRCCyN（南特中央理工学院机器人研究中心）；Nantes Atlantic University（南特大西洋大学）；University of Michigan（密歇根大学）；National Taiwan University of Science and Technology（台湾科技大学）
This paper presents three feedback controllers that achieve an asymptotically stable, periodic, and fast walking gait for a 3D (spatial) bipedal robot consisting of a torso, two legs, and passive (unactuated) point feet. The contact between the robot and the walking surface is assumed to inhibit yaw rotation. The studied robot has 8 DOF in the single support phase and 6 actuators. The interest of studying robots with point feet is that the robot's natural dynamics must be explicitly taken into account to achieve balance while walking. We use an extension of the method of virtual constraints an…
Hybrid systems, such as bipedal walkers, are challenging to control because of discontinuities in their nonlinear dynamics. Little can be predicted about the systems' evolution without modeling the guard conditions that govern transitions between hybrid modes, so even systems with reliable state sensing can be difficult to control. We propose an algorithm that allows for determining the hybrid mode of a system in real-time using data-driven analysis. The algorithm is used with data-driven dynamics identification to enable model predictive control based entirely on data. Two examples---a simula…
Stable bipedal walking is a key prerequisite for humanoid robots to reach their potential of being versatile helpers in our everyday environments. Bipedal walking is, however, a complex motion that requires the coordination of many degrees of freedom while it is also inherently unstable and sensitive to disturbances. The balance of a walking biped has to be constantly maintained. The most effective way of controlling balance are well timed and placed recovery steps -- capture steps -- that absorb the expense momentum gained from a push or a stumble. We present a bipedal gait generation framewo…
Ohio State University（俄亥俄州立大学）；Southern University of Science and Technology (SUSTech)（南方科技大学）
In this paper, a hierarchical and robust framework for learning bipedal locomotion is presented and successfully implemented on the 3D biped robot Digit built by Agility Robotics. We propose a cascade-structure controller that combines the learning process with intuitive feedback regulations. This design allows the framework to realize robust and stable walking with a reduced-dimension state and action spaces of the policy, significantly simplifying the design and reducing the sampling efficiency of the learning method. The inclusion of feedback regulation into the framework improves the robus…
This paper introduces a new approach to studying bipedal locomotion. The approach is based on magnetically actuated miniature robots. Building prototypes of bipedal locomotion machines has been very costly and overly complicated. We demonstrate that a magnetically actuated 0.3~gm robot, we call Big Foot, can be used to test fundamental ideas without necessitating very complex and expensive bipedal machines. We explore analytically and experimentally two age old questions in bipedal locomotion: 1. Can such robots be driven with pure hip actuation. 2. Is it better to use continuous or impulsive …
Visvesvaraya National Institute of Technology（维斯瓦瓦拉亚国家理工学院）；NASA-JPL, Caltech（加州理工学院NASA喷气推进实验室）
The selection of mobility modes for robot navigation consists of various trade-offs. Snake robots are ideal for traversing through constrained environments such as pipes, cluttered and rough terrain, whereas bipedal robots are more suited for structured environments such as stairs. Finally, quadruped robots are more stable than bipeds and can carry larger payloads than snakes and bipeds but struggle to navigate soft soil, sand, ice, and constrained environments. A reconfigurable robot can achieve the best of all worlds. Unfortunately, state-of-the-art reconfigurable robots rely on the rearrang…
In this paper, we propose a multi-domain control parameter learning framework that combines Bayesian Optimization (BO) and Hybrid Zero Dynamics (HZD) for locomotion control of bipedal robots. We leverage BO to learn the control parameters used in the HZD-based controller. The learning process is firstly deployed in simulation to optimize different control parameters for a large repertoire of gaits. Next, to tackle the discrepancy between the simulation and the real world, the learning process is applied on the physical robot to learn for corrections to the control parameters learned in simulat…
Ohio State University（俄亥俄州立大学）；Ottonomy Inc.（Ottonomy公司）
The popularity of mobile robots has been steadily growing, with these robots being increasingly utilized to execute tasks previously completed by human workers. For bipedal robots to see this same success, robust autonomous navigation systems need to be developed that can execute in real-time and respond to dynamic environments. These systems can be divided into three stages: perception, planning, and control. A holistic navigation framework for bipedal robots must successfully integrate all three components of the autonomous navigation problem to enable robust real-world navigation. In this p…
Walking motion planning based on Divergent Component of Motion (DCM) and Linear Inverted Pendulum Model (LIPM) is one of the alternatives that could be implemented to generate online humanoid robot gait trajectories. This algorithm requires different parameters to be adjusted. Herein, we developed a framework to attain optimal parameters to achieve a stable and energy-efficient trajectory for real robot's gait. To find the optimal trajectory, four cost functions representing energy consumption, the sum of joints velocity and applied torque at each lower limb joint of the robot, and a cost func…
Extended Hybrid Zero Dynamics for Bipedal Walking of the Knee-less Robot SLIDER
Robot Intelligence Lab, Dyson School of Design Engineering, Imperial College London, UK（机器人智能实验室，设计工程学院，伦敦帝国理工学院，英国）；Department of Aeronautics, Imperial College London, UK（航空系，伦敦帝国理工学院，英国）
Bipedal robots have received much attention because of the variety of motion maneuvers that they can produce, and the many applications they have in various areas including rehabilitation. One of these motion maneuvers is walking. In this study, we presented a framework for the trajectory optimization of a 5-link (planar) Biped Robot using hybrid optimization. The walking is modeled with two phases of single-stance (support) phase and the collision phase. The dynamic equations of the robot in each phase are extracted by the Lagrange method. It is assumed that the robot heel strike to the groun…
Collaborative Robotics and Intelligent Systems Institute, Oregon State University（俄勒冈州立大学协作机器人与智能系统研究所）
For legged robots to match the athletic capabilities of humans and animals, they must not only produce robust periodic walking and running, but also seamlessly switch between nominal locomotion gaits and more specialized transient maneuvers. Despite recent advancements in controls of bipedal robots, there has been little focus on producing highly dynamic behaviors. Recent work utilizing reinforcement learning to produce policies for control of legged robots have demonstrated success in producing robust walking behaviors. However, these learned policies have difficulty expressing a multitude of…
University of Lisbon（里斯本大学）；Instituto Superior Técnico（高等技术学院）
We present a solvable biped walking model based on an inverted pendulum with two massless articulated legs capable of walking on uneven floors and inclined planes. The stride of the two-legged robot results from the pendular motion of a standing leg and the articulated motion of a trailing leg. Gaiting is possible due to the alternating role of the legs, the standing and the trailing leg, and the conservation of energy of the pendular motion. The motion on uneven surfaces and inclined planes is possible by imposing the same maximal opening angle between the two legs in the transition between s…
Indian Institute of Technology (BHU) Varanasi（瓦拉纳西印度理工学院（BHU））；Ohio State University（俄亥俄州立大学）；Indian Institute of Science（印度科学学院）
In this work, we demonstrate robust walking in the bipedal robot Digit on uneven terrains by just learning a single linear policy. In particular, we propose a new control pipeline, wherein the high-level trajectory modulator shapes the end-foot ellipsoidal trajectories, and the low-level gait controller regulates the torso and ankle orientation. The foot-trajectory modulator uses a linear policy and the regulator uses a linear PD control law. As opposed to neural network-based policies, the proposed linear policy has only 13 learnable parameters, thereby not only guaranteeing sample efficient …
Machine learning algorithms have found several applications in the field of robotics and control systems. The control systems community has started to show interest towards several machine learning algorithms from the sub-domains such as supervised learning, imitation learning and reinforcement learning to achieve autonomous control and intelligent decision making. Amongst many complex control problems, stable bipedal walking has been the most challenging problem. In this paper, we present an architecture to design and simulate a planar bipedal walking robot(BWR) using a realistic robotics sim…
ETH Zurich（苏黎世联邦理工学院）；Max Planck Institute for Solar System Research（马克斯·普朗克太阳系研究所）
Celestial bodies such as the Moon and Mars are mainly covered by loose, granular soil, a notoriously challenging terrain to traverse with (wheeled) robotic systems. Here, we present experimental work on traversing steep, granular slopes with the dynamically walking quadrupedal robot SpaceBok. To adapt to the challenging environment, we developed passive-adaptive planar feet and optimized grouser pads to reduce sinkage and increase traction on planar and inclined granular soil. Single-foot experiments revealed that a large surface area of 110cm2 per foot reduces sinkage to an acceptable level e…
The most concentrated application of lower-limb rehabilitation exoskeleton (LLE) robot is that it can help paraplegics "re-walk". However, "walking" in daily life is more than just walking on flat ground with fixed gait. This paper focuses on variable gaits generation for LLE robot to adapt complex walking environment. Different from traditional gaits generator for biped robot, the generated gaits for LLEs should be comfortable to patients. Inspired by the pose graph optimization algorithm in SLAM, we propose a graph-based gait generation algorithm called gait graph optimization (GGO) to gener…
University of Aveiro（阿威罗大学）；University of Porto（波尔图大学）
This paper proposes a modular framework to generate robust biped locomotion using a tight coupling between an analytical walking approach and deep reinforcement learning. This framework is composed of six main modules which are hierarchically connected to reduce the overall complexity and increase its flexibility. The core of this framework is a specific dynamics model which abstracts a humanoid's dynamics model into two masses for modeling upper and lower body. This dynamics model is used to design an adaptive reference trajectories planner and an optimal controller which are fully parametric…
Robotics Institute, University of Michigan（密歇根大学机器人研究所）
We propose and experimentally demonstrate a reactive planning system for bipedal robots on unexplored, challenging terrains. The system consists of a low-frequency planning thread (5 Hz) to find an asymptotically optimal path and a high-frequency reactive thread (300 Hz) to accommodate robot deviation. The planning thread includes: a multi-layer local map to compute traversability for the robot on the terrain; an anytime omnidirectional Control Lyapunov Function (CLF) for use with a Rapidly Exploring Random Tree Star (RRT*) that generates a vector field for specifying motion between nodes; a s…
We present angle of attack control, a novel control strategy for a hip energized Penn Jerboa. The energetic losses from damping are counteracted by aligning most of the velocity at touchdown in the radial direction and the fore-aft velocity is controlled by using the hip torque to control to a target angular momentum. The control strategy results in highly asymmetric leg angle trajectories, thus avoiding the traction issues that plague hip actuated SLIP. Using a series of assumptions we find an analytical expression for the fixed points of an approximation to the hopping return map relating th…
