We present a sampling-based kinodynamic planning framework for a bipedal robot in complex environments. Unlike other footstep planner which typically plan footstep locations and the biped dynamics in separate steps, we handle both simultaneously. Three advantages of this approach are (1) the ability to differentiate alternate routes while selecting footstep locations based on the temporal duration of the route as determined by the Linear Inverted Pendulum Model dynamics, (2) the ability to perform collision checking through time so that collisions with moving obstacles are prevented without avoiding their entire trajectory, and (3) the ability to specify a minimum forward velocity for the biped. To generate a dynamically consistent description of the walking behavior, we exploit the Phase Space Planner. To plan a collision free route toward the goal, we adapt planning strategies from non-holonomic wheeled robots to gather a sequence of inputs for the PSP. This allows us to efficiently approximate dynamic and kinematic constraints on bipedal motion, to apply a sampling based planning algorithms, and to use the Dubin's path as the steering method to connect two points in the configuration space. The results of the algorithm are sent to a Whole Body Controller to generate full body dynamic walking behavior.
This paper combines episodic learning and control barrier functions in the setting of bipedal locomotion. The safety guarantees that control barrier functions provide are only valid with perfect model knowledge; however, this assumption cannot be met on hardware platforms. To address this, we utilize the notion of projection-to-state safety paired with a machine learning framework in an attempt to learn the model uncertainty as it affects the barrier functions. The proposed approach is demonstrated both in simulation and on hardware for the AMBER-3M bipedal robot in the context of the stepping…
Robust and energy-efficient bipedal locomotion in robotics is still a challenging topic. In order to address issues in this field, we can take inspiration from nature, by studying human locomotion. The Spring-Loaded Inverted Pendulum (SLIP) model has shown to be a good model for this purpose. However, the human musculoskeletal system enables us to actively modulate leg stiffness, for example when walking in rough terrain with irregular and unexpected height variations of the walking surface. This ability of varying leg stiffness is not considered in conventional SLIP-based models, and therefor…
University of Zurich（苏黎世大学）；Ghent University（根特大学）；Fondazione Santa Lucia（圣卢西亚基金会）
Bipedal locomotion is a phenomenon that still eludes a fundamental and concise mathematical understanding. Conceptual models that capture some relevant aspects of the process exist but their full explanatory power is not yet exhausted. In the current study, we introduce the robustness criterion which defines the conditions for stable locomotion when steps are taken with imprecise angle of attack. Intuitively, the necessity of a higher precision indicates the difficulty to continue moving with a given gait. We show that the spring-loaded inverted pendulum model, under the robustness criterion, …
Delft University of Technology（代尔夫特理工大学）；California Institute of Technology（加州理工学院）
Humans are able to negotiate downstep behaviors -- both planned and unplanned -- with remarkable agility and ease. The goal of this paper is to systematically study the translation of this human behavior to bipedal walking robots, even if the morphology is inherently different. Concretely, we begin with human data wherein planned and unplanned downsteps are taken. We analyze this data from the perspective of reduced-order modeling of the human, encoding the center of mass (CoM) kinematics and contact forces, which allows for the translation of these behaviors into the corresponding reduced-ord…
Collaborative Robotics and Intelligent Systems Institute, Oregon State University（俄勒冈州立大学协作机器人与智能系统研究所）；Agility Robotics
Recent work has demonstrated the success of reinforcement learning (RL) for training bipedal locomotion policies for real robots. This prior work, however, has focused on learning joint-coordination controllers based on an objective of following joint trajectories produced by already available controllers. As such, it is difficult to train these approaches to achieve higher-level goals of legged locomotion, such as simply specifying the desired end-effector foot movement or ground reaction forces. In this work, we propose an approach for integrating knowledge of the robot system into RL to all…
ETH Zürich（苏黎世联邦理工学院）；California Institute of Technology（加州理工学院）
The ability to generate dynamic walking in real-time for bipedal robots with input constraints and underactuation has the potential to enable locomotion in dynamic, complex and unstructured environments. Yet, the high-dimensional nature of bipedal robots has limited the use of full-order rigid body dynamics to gaits which are synthesized offline and then tracked online. In this work we develop an online nonlinear model predictive control approach that leverages the full-order dynamics to realize diverse walking behaviors. Additionally, this approach can be coupled with gaits synthesized offlin…
TOLEBI: Learning Fault-Tolerant Bipedal Locomotion via Online Status Estimation and Fallibility Rewards
Department of Intelligence and Information, Graduate School of Convergence Science and Technology, Seoul National University（智能信息系，融合科学与技术研究生院，首尔国立大学）；Artificial Intelligence Institute (AIIS), Seoul National University（人工智能研究院（AIIS），首尔国立大学）；Advanced Institutes of Convergence Technology (AICT), Suwon（融合技术高级研究院（AICT），Suwon）
Beijing Research Institute of UBTECH Robotics（优必选机器人北京研究院）；Tsinghua University（清华大学）
The planning of whole-body motion and step time for bipedal locomotion is constructed as a model predictive control (MPC) problem, in which a sequence of optimization problems needs to be solved online. While directly solving these problems is extremely time-consuming, we propose a predictive gait synthesizer to offer immediate solutions. Based on the full-dimensional model, a library of gaits with different speeds and periods is first constructed offline. Then the proposed gait synthesizer generates real-time gaits at 1kHz by synthesizing the gait library based on the online prediction of cen…
This work presents Neural Gaits, a method for learning dynamic walking gaits through the enforcement of set invariance that can be refined episodically using experimental data from the robot. We frame walking as a set invariance problem enforceable via control barrier functions (CBFs) defined on the reduced-order dynamics quantifying the underactuated component of the robot: the zero dynamics. Our approach contains two learning modules: one for learning a policy that satisfies the CBF condition, and another for learning a residual dynamics model to refine imperfections of the nominal model. Im…
The ability to generate robust walking gaits on bipedal robots is key to their successful realization on hardware. To this end, this work extends the method of Hybrid Zero Dynamics (HZD) -- which traditionally only accounts for locomotive stability via periodicity constraints under perfect impact events -- through the inclusion of the saltation matrix with a view toward synthesizing robust walking gaits. By jointly minimizing the norm of the extended saltation matrix and the torque of the robot directly in the gait generation process, we demonstrate that the synthesized gaits are more robust t…
University of Notre Dame（圣母大学）；Robotics at Google（谷歌机器人部门）
Reinforcement Learning (RL) has witnessed great strides for quadruped locomotion, with continued progress in the reliable sim-to-real transfer of policies. However, it remains a challenge to reuse a policy on another robot, which could save time for retraining. In this work, we present a framework for zero-shot policy retargeting wherein diverse motor skills can be transferred between robots of different shapes and sizes. The new framework centers on a planning-and-control pipeline that systematically integrates RL and Model Predictive Control (MPC). The planning stage employs RL to generate a…
State-of-the-art approaches to legged locomotion are widely dependent on the use of models like the linear inverted pendulum (LIP) and the spring-loaded inverted pendulum (SLIP), popular because their simplicity enables a wide array of tools for planning, control, and analysis. However, they inevitably limit the ability to execute complex tasks or agile maneuvers. In this work, we aim to automatically synthesize models that remain low-dimensional but retain the capabilities of the high-dimensional system. For example, if one were to restore a small degree of complexity to LIP, SLIP, or a simil…
When legged robots impact their environment, they undergo large changes in their velocities in a small amount of time. Measuring and applying feedback to these velocities is challenging, and is further complicated due to uncertainty in the impact model and impact timing. This work proposes a general framework for adapting feedback control during impact by projecting the control objectives to a subspace that is invariant to the impact event. The resultant controller is robust to uncertainties in the impact event while maintaining maximum control authority over the impact invariant subspace. We …
University of Tehran（德黑兰大学）；École polytechnique fédérale de Lausanne（洛桑联邦理工学院）
In this paper, we present a novel control architecture for the online adaptation of bipedal locomotion on inclined obstacles. In particular, we introduce a novel, cost-effective, and versatile foot sensor to detect the proximity of the robot's feet to the ground (bump sensor). By employing this sensor, feedback controllers are implemented to reduce the impact forces during the transition of the swing to stance phase or steeping on inclined unseen obstacles. Compared to conventional sensors based on contact reaction force, this sensor detects the distance to the ground or obstacles before the f…
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Hong Kong Institute of Science & Innovation, Chinese Academy of Sciences（中国科学院香港创新研究院）；University of Chinese Academy of Sciences（中国科学院大学）
Swing arms have an irreplaceable role in promoting highly dynamic locomotion on bipedal robots by a larger angular momentum control space from the viewpoint of biomechanics. Few bipedal robots utilize swing arms and its redundancy characteristic of multiple degrees of freedom due to the lack of appropriate locomotion control strategies to perfectly integrate modeling and control. This paper presents a kind of control strategy by modeling the bipedal robot as a flywheel-spring loaded inverted pendulum (F-SLIP) to extract characteristics of swing arms and using the whole-body controller (WBC) to…
In this paper, previous works on the Model Predictive Control (MPC) and the Divergent Component of Motion (DCM) for bipedal walking control are extended. To this end, we employ a single MPC which uses a combination of Center of Pressure (CoP) manipulation, step adjustment, and Centroidal Moment Pivot (CMP) modulation to design a robust walking controller. Furthermore, we exploit the concept of time-varying DCM to generalize our walking controller for walking in uneven surfaces. Using our scheme, a general and robust walking controller is designed which can be implemented on robots with differe…
Sun Yat-sen University（中山大学）；University of Oxford（牛津大学）
Controlling a non-statically bipedal robot is challenging due to the complex dynamics and multi-criterion optimization involved. Recent works have demonstrated the effectiveness of deep reinforcement learning (DRL) for simulation and physical robots. In these methods, the rewards from different criteria are normally summed to learn a single value function. However, this may cause the loss of dependency information between hybrid rewards and lead to a sub-optimal policy. In this work, we propose a novel reward-adaptive reinforcement learning for biped locomotion, allowing the control policy to …
Learning Terrain Aware Bipedal Locomotion via Reduced Dimensional Perceptual Representations
Electrical and Computer Engineering, The Ohio State University（电气与计算机工程系，俄亥俄州立大学）；Mechanical and Aerospace Engineering, The Ohio State University（机械与航空航天工程系，俄亥俄州立大学）
LIPM-Guided Reinforcement Learning for Stable and Perceptive Locomotion in Bipedal Robots
School of Automation and Intelligent Manufacturing (AIM), Southern University of Science and Technology（自动化与智能制造学院，南方科技大学）；Department of Civil and Environmental Engineering, The Hong Kong University of Science and Technology（土木与环境工程系，香港科技大学）；Department of Mechanical Engineering, The University of Hong Kong（机械工程系，香港大学）；Zhejiang University-University of Illinois Urbana-Champaign Institute (ZJUI), Zhejiang University（浙江大学-伊利诺伊大学厄巴纳-香槟分校联合研究所，浙江大学）；LimX Dynamics, Shenzhen, China（深圳LimX动力有限公司）
Collaborative Robotics and Intelligent Systems Institute, Oregon State University（俄勒冈州立大学协作机器人与智能系统研究所）
Recent work on sim-to-real learning for bipedal locomotion has demonstrated new levels of robustness and agility over a variety of terrains. However, that work, and most prior bipedal locomotion work, have not considered locomotion under a variety of external loads that can significantly influence the overall system dynamics. In many applications, robots will need to maintain robust locomotion under a wide range of potential dynamic loads, such as pulling a cart or carrying a large container of sloshing liquid, ideally without requiring additional load-sensing capabilities. In this work, we ex…
While quadruped robots usually have good stability and load capacity, bipedal robots offer a higher level of flexibility / adaptability to different tasks and environments. A multi-modal legged robot can take the best of both worlds. In this paper, we propose a multi-modal locomotion framework that is composed of a hand-crafted transition motion and a learning-based bipedal controller -- learnt by a novel algorithm called Automated Residual Reinforcement Learning. This framework aims to endow arbitrary quadruped robots with the ability to walk bipedally. In particular, we 1) design an addition…
Nonlinear Model Predictive Control for Robust Bipedal Locomotion: Exploring Angular Momentum and CoM Height Changes
School of Power and Mechanical Engineering, Wuhan University（武汉大学动力与机械学院）；Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳人工智能与机器人研究院）；School of Mechanical Engineering, University of Leeds（利兹大学机械工程学院）；School of Informatics, The University of Edinburgh（爱丁堡大学信息学院）；Humanoid and Human Centered Mechatronics Research Line, Istituto Italiano di Tecnologia（意大利技术研究院人形与以人为中心的机电一体化研究线）
Collaborative Robotics and Intelligent Systems Institute（协作机器人与智能系统研究院）；Oregon State University（俄勒冈州立大学）
Controlling a non-statically stable biped is a difficult problem largely due to the complex hybrid dynamics involved. Recent work has demonstrated the effectiveness of reinforcement learning (RL) for simulation-based training of neural network controllers that successfully transfer to real bipeds. The existing work, however, has primarily used simple memoryless network architectures, even though more sophisticated architectures, such as those including memory, often yield superior performance in other RL domains. In this work, we consider recurrent neural networks (RNNs) for sim-to-real biped …
Bipedal-Walking-Dynamics Model on Granular Terrains
Department of Mechanical and Aerospace Engineering, Rutgers University（罗格斯大学机械与航空航天工程系）；Bridgewater Raritan High School（布里奇沃特-拉里坦高中）；State Key Lab of Fluid Power Transmission and Control and the School of Mechanical Engineering, Zhejiang University（浙江大学流体动力传动与控制国家重点实验室和机械工程学院）
