Unknown dynamic load carrying is one important practical application for quadruped robots. Such a problem is non-trivial, posing three major challenges in quadruped locomotion control. First, how to model or represent the dynamics of the load in a generic manner. Second, how to make the robot capture the dynamics without any external sensing. Third, how to enable the robot to interact with load handling the mutual effect and stabilizing the load. In this work, we propose a general load modeling approach called load characteristics modeling to capture the dynamics of the load. We integrate this proposed modeling technique and leverage recent advances in Reinforcement Learning (RL) based locomotion control to enable the robot to infer the dynamics of load movement and interact with the load indirectly to stabilize it and realize the sim-to-real deployment to verify its effectiveness in real scenarios. We conduct extensive comparative simulation experiments to validate the effectiveness and superiority of our proposed method. Results show that our method outperforms other methods in sudden load resistance, load stabilizing and locomotion with heavy load on rough terrain. \href{https://leixinjonaschang.github.io/leggedloadadapt.github.io/}{Project Page}.
TRANS: Terrain-aware Reinforcement Learning for Agile Navigation of Quadruped Robots under Social Interactions
Department of Robotics, Graduate School of Engineering, Tohoku University（东邦大学机器人学系）；Laboratory for Intelligent Decision and Autonomous Robots, Woodruff School of Mechanical Engineering, Georgia Institute of Technology（佐治亚理工学院智能决策与自主机器人实验室）
SF-TIM: A Simple Framework for Enhancing Quadrupedal Robot Jumping Agility by Combining Terrain Imagination and Measurement
State Key Laboratory of Modern Optical Instrumentation, Zhejiang University, China（浙江大学现代光学仪器国家重点实验室）；School of Artificial Intelligence and Robotics and National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, China（湖南大学人工智能与机器人学院和机器人视觉感知与控制技术国家工程研究中心）；State Key Laboratory of Industrial Control Technology, Zhejiang University, China（浙江大学工业控制技术国家重点实验室）；DeepRobotics Co. Ltd., China（深睿科技有限公司）
University of British Columbia（不列颠哥伦比亚大学）；NVIDIA（英伟达）；University of Toronto（多伦多大学）；Vector Institute（向量研究院）
Understanding the gap between simulation and reality is critical for reinforcement learning with legged robots, which are largely trained in simulation. However, recent work has resulted in sometimes conflicting conclusions with regard to which factors are important for success, including the role of dynamics randomization. In this paper, we aim to provide clarity and understanding on the role of dynamics randomization in learning robust locomotion policies for the Laikago quadruped robot. Surprisingly, in contrast to prior work with the same robot model, we find that direct sim-to-real transf…
Fine-Tuning Hard-to-Simulate Objectives for Quadruped Locomotion: A Case Study on Total Power Saving
Institute for Interdisciplinary Information Sciences, Tsinghua University（清华大学交叉信息研究院）；Shanghai AI Lab（上海人工智能实验室）；Shanghai Qi Zhi Institute（上海期智研究院）；Department of Electronic Engineering, Tsinghua University（清华大学电子工程系）
ZSL-RPPO: Zero-Shot Learning for Quadrupedal Locomotion in Challenging Terrains using Recurrent Proximal Policy Optimization
Huawei Technologies（华为技术有限公司）；Huawei Munich Research Center（华为慕尼黑研究中心）；University College London（伦敦大学学院）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；East China Normal University（华东师范大学）
Designing agile locomotion for quadruped robots often requires extensive expertise and tedious manual tuning. In this paper, we present a system to automate this process by leveraging deep reinforcement learning techniques. Our system can learn quadruped locomotion from scratch using simple reward signals. In addition, users can provide an open loop reference to guide the learning process when more control over the learned gait is needed. The control policies are learned in a physics simulator and then deployed on real robots. In robotics, policies trained in simulation often do not transfer t…
Shanghai Jiao Tong University（上海交通大学）；Digital Brain Lab（数字大脑实验室）；ShanghaiTech University（上海科技大学）；University College London（伦敦大学学院）
Deep reinforcement learning has recently emerged as an appealing alternative for legged locomotion over multiple terrains by training a policy in physical simulation and then transferring it to the real world (i.e., sim-to-real transfer). Despite considerable progress, the capacity and scalability of traditional neural networks are still limited, which may hinder their applications in more complex environments. In contrast, the Transformer architecture has shown its superiority in a wide range of large-scale sequence modeling tasks, including natural language processing and decision-making pro…
Meta AI；University of California, Berkeley（加州大学伯克利分校）
Reinforcement learning (RL) has become a promising approach to developing controllers for quadrupedal robots. Conventionally, an RL design for locomotion follows a position-based paradigm, wherein an RL policy outputs target joint positions at a low frequency that are then tracked by a high-frequency proportional-derivative (PD) controller to produce joint torques. In contrast, for the model-based control of quadrupedal locomotion, there has been a paradigm shift from position-based control to torque-based control. In light of the recent advances in model-based control, we explore an alternati…
DiffSim2Real: Deploying Quadrupedal Locomotion Policies Purely Trained in Differentiable Simulation
Robotic Systems Lab, ETH Zürich（苏黎世联邦理工学院机器人系统实验室）；Georgia Institute of Technology（佐治亚理工学院）；The AI Institute（人工智能研究所）；Institute for Research in Technology, ICAI School of Engineering, Comillas Pontifical University（科米利亚斯主教大学技术研究所（ICAI工程学院））
Ecole Polytechnique Federale de Lausanne（洛桑联邦理工学院）
In this letter, we present a method for integrating central pattern generators (CPGs), i.e. systems of coupled oscillators, into the deep reinforcement learning (DRL) framework to produce robust and omnidirectional quadruped locomotion. The agent learns to directly modulate the intrinsic oscillator setpoints (amplitude and frequency) and coordinate rhythmic behavior among different oscillators. This approach also allows the use of DRL to explore questions related to neuroscience, namely the role of descending pathways, interoscillator couplings, and sensory feedback in gait generation. We trai…
Locomotion has seen dramatic progress for walking or running across challenging terrains. However, robotic quadrupeds are still far behind their biological counterparts, such as dogs, which display a variety of agile skills and can use the legs beyond locomotion to perform several basic manipulation tasks like interacting with objects and climbing. In this paper, we take a step towards bridging this gap by training quadruped robots not only to walk but also to use the front legs to climb walls, press buttons, and perform object interaction in the real world. To handle this challenging optimiza…
Istituto Italiano di Tecnologia（意大利理工学院）；Università di Trento（特伦托大学）；PAL Robotics（PAL机器人公司）
The Whole-Body Locomotion Framework (WoLF) is an end-to-end software suite devoted to the loco-manipulation of quadruped robots. WoLF abstracts the complexity of planning and control of quadrupedal robot hardware into a simple to use and robust software that can be connected through multiple tele-operation devices to different quadruped robot models. Furthermore, WoLF allows controlling mounted devices, such as arms or pan-tilt cameras, jointly with the quadrupedal platform. In this short paper, we introduce the main features of WoLF and its overall software architecture.
In this paper, we present a effective state estimation algorithm that combined with various sensors information (Inertial measurement unit, joints encoder, camera and LIDAR)
Shenzhen Institute of Artificial Intelligence and Robotics for Society (AIRS)（深圳市人工智能与机器人研究院）；Institute of Robotics and Intelligent Manufacturing (IRIM), The Chinese University of Hong Kong (CUHK)（香港中文大学机器人与智能制造研究院）；George W. Woodruff School of Mechanical Engineering, Georgia Institute of Technology（佐治亚理工学院乔治·W·伍德拉夫机械工程学院）；Beijing Institute for General Artificial Intelligence（北京通用人工智能研究院）
Dynamic quadrupedal locomotion over rough terrains reveals remarkable progress over the last few decades. Small-scale quadruped robots are adequately flexible and adaptable to traverse uneven terrains along sagittal direction, such as slopes and stairs. To accomplish autonomous locomotion navigation in complex environments, spinning is a fundamental yet indispensable functionality for legged robots. However, spinning behaviors of quadruped robots on uneven terrain often exhibit position drifts. Motivated by this problem, this study presents an algorithmic method to enable accurate spinning mot…
MASQ: Multi-Agent Reinforcement Learning for Single Quadruped Robot Locomotion
Harbin Institute of Technology Shenzhen（哈尔滨工业大学（深圳））；School of Mechanical Engineering and Automation（机械工程与自动化学院）；Guangdong Key Laboratory of Intelligent Morphing Mechanisms and Adaptive Robotics（广东省智能变胞机构与自适应机器人重点实验室）
Dynamic Quadrupedal Legged and Aerial Locomotion via Structure Repurposing
Department of Electrical and Computer Engineering, Northeastern University（电气与计算机工程系，东北大学）；Department of Aerospace Engineering, California Institute of Technology（航空航天工程系，加州理工学院）
Load-bearing Assessment for Safe Locomotion of Quadruped Robots on Collapsing Terrain
Dynamic Legged Systems Lab, Istituto Italiano di Tecnologia (IIT)（动态腿系统实验室，意大利技术研究院）；Department of Mechanical Engineering, São Carlos School of Engineering (EESC), University of São Paulo (USP)（机械工程系，圣卡洛斯工程学院（EESC），圣保罗大学）
We present a versatile nonlinear model predictive control (NMPC) formulation for quadrupedal locomotion. Our formulation jointly optimizes a base trajectory and a set of footholds over a finite time horizon based on simplified dynamics models. We leverage second-order sensitivity analysis and a sparse Gauss-Newton (SGN) method to solve the resulting optimal control problems. We further describe our ongoing effort to verify our approach through simulation and hardware experiments. Finally, we extend our locomotion framework to deal with challenging tasks that comprise gap crossing, movement on …
Saving the Limping: Fault-tolerant Quadruped Locomotion via Reinforcement Learning
NVIDIA AI Technology Centre (NVAITC)（英伟达人工智能技术中心（NVAITC））；School of Computer Science and Engineering, Nanyang Technological University（南洋理工大学计算机科学与工程学院）；Coventry University（考文垂大学）；Mahindra University（马恒达大学）
University of Pisa（比萨大学）；Research Center “E. Piaggio” of the University of Pisa（比萨大学“E.皮亚乔”研究中心）
In this paper, we aim to improve the robustness of dynamic quadrupedal locomotion through two aspects: 1) fast model predictive foothold planning, and 2) applying LQR to projected inverse dynamic control for robust motion tracking. In our proposed planning and control framework, foothold plans are updated at 400 Hz considering the current robot state and an LQR controller generates optimal feedback gains for motion tracking. The LQR optimal gain matrix with non-zero off-diagonal elements leverages the coupling of dynamics to compensate for system underactuation. Meanwhile, the projected invers…
Shenzhen Institute of Artificial Intelligence and Robotics for Society (AIRS)（深圳人工智能与机器人研究院）；Georgia Institute of Technology（佐治亚理工学院）；Hong Kong University of Science and Technology（香港科技大学）；Southern University of Science and Technology (SUSTech)（南方科技大学）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
Quadruped robots manifest great potential to traverse rough terrains with payload. Numerous traditional control methods for legged dynamic locomotion are model-based and exhibit high sensitivity to model uncertainties and payload variations. Therefore, high-performance model parameter estimation becomes indispensable. However, the inertia parameters of payload are usually unknown and dynamically changing when the quadruped robot is deployed in versatile tasks. To address this problem, online identification of the inertia parameters and the Center of Mass (CoM) position of the payload for the q…
University of Bonn（波恩大学）；Dexterity Inc.（Dexterity公司）
Fast and versatile locomotion can be achieved with wheeled quadruped robots that drive quickly on flat terrain, but are also able to overcome challenging terrain by adapting their body pose and by making steps. In this paper, we present a state estimation approach for four-legged robots with non-steerable wheels that enables hybrid driving-stepping locomotion capabilities. We formulate a Kalman Filter (KF) for state estimation that integrates driven wheels into the filter equations and estimates the robot state (position and velocity) as well as the contribution of driving with wheels to the a…
Istituto Italiano di Tecnologia (IIT)（意大利理工学院）；LAAS-CNRS（法国国家科学研究中心自动化系统实验室）
We present experimental results using a passive whole-body control approach for quadruped robots that achieves dynamic locomotion while compliantly balancing the robot's trunk. We formulate the motion tracking as a Quadratic Program (QP) that takes into account the full robot rigid body dynamics, the actuation limits, the joint limits and the contact interaction. We analyze the controller's robustness against inaccurate friction coefficient estimates and unstable footholds, as well as its capability to redistribute the load as a consequence of enforcing actuation limits. Additionally, we prese…
Minimizing Acoustic Noise: Enhancing Quiet Locomotion for Quadruped Robots in Indoor Applications
Department of Computer Science and Engineering, Shanghai Jiao Tong University（计算机科学与工程系，上海交通大学）；Department of Automation, Shanghai Jiao Tong University（自动化系，上海交通大学）；MoE Key Lab of Artificial Intelligence and AI Institute, Shanghai Jiao Tong University（人工智能与AI研究所，上海交通大学）
Evaluation of an Actuated Spine in Agile Quadruped Locomotion
Department of Computer Science, Technical University of Darmstadt（德意志技术大学达姆施塔特分校计算机科学系）；IDEAS Research Institute（IDEAS研究所）；Institute of Robotics and Machine Intelligence, Poznan University of Technology（波兹南技术大学机器人与机器智能研究所）
Two-layer adaptive trajectory tracking controller for quadruped robots on slippery terrains
Institute of Computer Science, Foundation for Research and Technology–Hellas（希腊基础研究与技术基金会计算机科学研究所）；Hellenic Mediterranean University（希腊地中海大学）；University of Crete（克里特大学）
Terrain-Aware Quadrupedal Locomotion via Reinforcement Learning
Tencent Robotics X（腾讯Robotics X）；The Chinese University of Hong Kong（香港中文大学）；Southern University of Science and Technology（南方科技大学）；University of Alberta（阿尔伯塔大学）
