When legged robots perform agile movements, traditional RGB cameras often produce blurred images, posing a challenge for rapid perception. Event cameras have emerged as a promising solution for capturing rapid perception and coping with challenging lighting conditions thanks to their low latency, high temporal resolution, and high dynamic range. However, integrating event cameras into agile-legged robots is still largely unexplored. Notably, no dataset including event cameras has yet been developed for the context of agile quadruped robots. To bridge this gap, we introduce CEAR, a dataset comprising data from an event camera, an RGB-D camera, an IMU, a LiDAR, and joint encoders, all mounted on a dynamic quadruped, Mini Cheetah robot. This comprehensive dataset features more than 100 sequences from real-world environments, encompassing various indoor and outdoor environments, different lighting conditions, a range of robot gaits (e.g., trotting, bounding, pronking), as well as acrobatic movements like backflip. To our knowledge, this is the first event camera dataset capturing the dynamic and diverse quadruped robot motions under various setups, developed to advance research in rapid perception for quadruped robots.
Robotics at Google（谷歌机器人）；Georgia Institute of Technology（佐治亚理工学院）
Developing agile behaviors for legged robots remains a challenging problem. While deep reinforcement learning is a promising approach, learning truly agile behaviors typically requires tedious reward shaping and careful curriculum design. We formulate agile locomotion as a multi-stage learning problem in which a mentor guides the agent throughout the training. The mentor is optimized to place a checkpoint to guide the movement of the robot's center of mass while the student (i.e. the robot) learns to reach these checkpoints. Once the student can solve the task, we teach the student to perform …
Legged robots pose one of the greatest challenges in robotics. Dynamic and agile maneuvers of animals cannot be imitated by existing methods that are crafted by humans. A compelling alternative is reinforcement learning, which requires minimal craftsmanship and promotes the natural evolution of a control policy. However, so far, reinforcement learning research for legged robots is mainly limited to simulation, and only few and comparably simple examples have been deployed on real systems. The primary reason is that training with real robots, particularly with dynamically balancing systems, is …
Evaluation of an Actuated Spine in Agile Quadruped Locomotion
Department of Computer Science, Technical University of Darmstadt（德意志技术大学达姆施塔特分校计算机科学系）；IDEAS Research Institute（IDEAS研究所）；Institute of Robotics and Machine Intelligence, Poznan University of Technology（波兹南技术大学机器人与机器智能研究所）
Learning Quadruped Locomotion using Bio-Inspired Neural Networks with Intrinsic Rhythmicity
Shenzhen Amigaga Technology Co Ltd.（深圳阿米加加科技有限公司）；Sun Yat-sen University（中山大学）；Shenyang Institute of Automation, Chinese Academy of Sciences（中国科学院沈阳自动化研究所）；University College London（伦敦大学学院）
Developing controllers for agile locomotion is a long-standing challenge for legged robots. Reinforcement learning (RL) and Evolution Strategy (ES) hold the promise of automating the design process of such controllers. However, dedicated and careful human effort is required to design training environments to promote agility. In this paper, we present a multi-agent learning system, in which a quadruped robot (protagonist) learns to chase another robot (adversary) while the latter learns to escape. We find that this adversarial training process not only encourages agile behaviors but also effect…
We present Stanford Pupper, an easily-replicated open source quadruped robot designed specifically as a benchmark platform for legged robotics research. The robot features torque-controllable brushless motors with high specific power that enable testing of impedance and torque-based machine learning and optimization control approaches. Pupper can be built from the ground up in under 8 hours for a total cost under $2000, with all components either easily purchased or 3D printed. To rigorously compare control approaches, we introduce two benchmarks, Sprint and Scramble with a leader board mainta…
TRAIN: Terrain-Aware Interactive Navigation for Quadruped Robots via Reinforcement Learning
Department of Robotics, Graduate School of Engineering, Tohoku University（东邦大学机器人学系）；Laboratory for Intelligent Decision and Autonomous Robots, Woodruff School of Mechanical Engineering, Georgia Institute of Technology（佐治亚理工学院智能决策与自主机器人实验室）
KYON: Semi-Modular Wheel-Legged Quadruped With Agile Bimanual Capability
Humanoids and Human Centered Mechatronics, Fondazione Istituto Italiano di Tecnologia（人机一体化机械系统，意大利理工研究院）；Advanced Robotics Facility, Fondazione Istituto Italiano di Tecnologia（先进机器人设施，意大利理工研究院）
Designing agile locomotion for quadruped robots often requires extensive expertise and tedious manual tuning. In this paper, we present a system to automate this process by leveraging deep reinforcement learning techniques. Our system can learn quadruped locomotion from scratch using simple reward signals. In addition, users can provide an open loop reference to guide the learning process when more control over the learned gait is needed. The control policies are learned in a physics simulator and then deployed on real robots. In robotics, policies trained in simulation often do not transfer t…
Robots operating in human environments need various skills, like slow and fast walking, turning, side-stepping, and many more. However, building robot controllers that can exhibit such a large range of behaviors is a challenging problem that requires tedious investigation for every task. We present a unified model-based control algorithm for imitating different animal gaits without expensive simulation training or real-world fine-tuning. Our method consists of stance and swing leg controllers using a centroidal dynamics model augmented with online adaptation techniques. We also develop a whole…
NASA Jet Propulsion Laboratory, California Institute of Technology（美国国家航空航天局喷气推进实验室，加州理工学院）；University of Goettingen, University Medical Center Goettingen（哥廷根大学，哥廷根大学医学中心）；Institute for Robotics and Intelligent Machines, Georgia Institute of Technology（佐治亚理工学院机器人与智能机器研究所）；Swiss Federal Institute of Technology (ETH Zürich)（苏黎世联邦理工学院）；California Institute of Technology（加州理工学院）
Legged robots can traverse a wide variety of terrains, some of which may be challenging for wheeled robots, such as stairs or highly uneven surfaces. However, quadruped robots face stability challenges on slippery surfaces. This can be resolved by adjusting the robot's locomotion by switching to more conservative and stable locomotion modes, such as crawl mode (where three feet are in contact with the ground always) or amble mode (where one foot touches down at a time) to prevent potential falls. To tackle these challenges, we propose an approach to learn a model from past robot experience for…
University of Edinburgh（爱丁堡大学）；University College London（伦敦大学学院）
This work developed a kernel-based residual learning framework for quadrupedal robotic locomotion. Initially, a kernel neural network is trained with data collected from an MPC controller. Alongside a frozen kernel network, a residual controller network is trained via reinforcement learning to acquire generalized locomotion skills and resilience against external perturbations. With this proposed framework, a robust quadrupedal locomotion controller is learned with high sample efficiency and controllability, providing omnidirectional locomotion at continuous velocities. Its versatility and robu…
National University of Defense Technology（国防科技大学）；College of Intelligence Science and Technology（智能科学与技术学院）；Robotics Research Center（机器人研究中心）
In this paper, we propose a robust controller that achieves natural and stably fast locomotion on a real blind quadruped robot. With only proprioceptive information, the quadruped robot can move at a maximum speed of 10 times its body length, and has the ability to pass through various complex terrains. The controller is trained in the simulation environment by model-free reinforcement learning. In this paper, the proposed loose neighborhood control architecture not only guarantees the learning rate, but also obtains an action network that is easy to transfer to a real quadruped robot. Our res…
PIE: Parkour with Implicit-Explicit Learning Framework for Legged Robots
Institute of Cyber-Systems and Control, Zhejiang University（浙江大学控制科学与工程学院 cyber-system 研究所）；State Key Laboratory of Industrial Control Technology（工业控制技术国家重点实验室）
M-SEVIQ: A Multi-band Stereo Event Visual-Inertial Quadruped-based Dataset for Perception under Rapid Motion and Challenging Illumination
Shanghai Key Laboratory of Navigation & Location Based Services, Shanghai Jiao Tong University（上海导航与基于位置的服务关键实验室，上海交通大学）；Beijing Aerospace Microsystem and Information Technology Research Institute（北京航天微系统与信息技术研究所）
SoloParkour: Constrained Reinforcement Learning for Visual Locomotion from Privileged Experience
LAAS-CNRS（法国国家科学研究中心自动化与系统分析实验室）；Université de Toulouse（图卢兹大学）；CNRS（法国国家科学研究中心）；New York University（纽约大学）；Artificial and Natural Intelligence Toulouse Institute（图卢兹人工与自然智能研究所）
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）
Animals locomote robustly and agile, albeit significant sensorimotor delays of their nervous system. The sensorimotor control of legged robots is implemented with much higher frequencies-often in the kilohertz range-and sensor and actuator delays in the low millisecond range. But especially at harsh impacts with unknown touch-down timing, legged robots show unstable controller behaviors, while animals are seemingly not impacted. Here we examine this discrepancy and suggest a hybrid robotic leg and controller design. We implemented a physical, parallel joint compliance dimensioned in combinatio…
The quadruped robot is a versatile mobile platform with potential ability for high payload carrying. However, most of the existing quadruped robots aim at high maneuverability, highly dynamic and agile locomotion. In spite of this, payload carrying is still an indispensable ability for the quadruped robots. Design of a quadruped robot with high payload capacity is yet deeply explored. In this study, a 50 kg electrically-actuated quadruped robot, Kirin, is presented to leverage the payload carrying capability. Kirin is an characterized with prismatic quasi-direct-drive (QDD) leg. This mechanism…
Robots with Attitude: Singularity-Free Quaternion-Based Model-Predictive Control for Agile Legged Robots
Center for Robotics and Biosystems, Northwestern University（西北大学机器人与生物系统中心）；Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）；Department of Mechanical Engineering, Carnegie Mellon University（卡内基梅隆大学机械工程系）
Hybrid Dynamics Modeling and Trajectory Planning for a Cable-Trailer System with a Quadruped Robot
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（人工智能与自动化学院，华中科技大学）；Department of Data Science, City University of Hong Kong（数据科学系，城市大学）；Embodied AI Center of Shanghai AI Lab（上海人工智能实验室具身智能中心）
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Tandon School of Engineering, New York University（纽约大学坦登工程学院）；Oxford Robotics Institute, University of Oxford（牛津大学牛津机器人研究所）
Implementing dynamic locomotion behaviors on legged robots requires a high-quality state estimation module. Especially when the motion includes flight phases, state-of-the-art approaches fail to produce reliable estimation of the robot posture, in particular base height. In this paper, we propose a novel approach for combining visual-inertial odometry (VIO) with leg odometry in an extended Kalman filter (EKF) based state estimator. The VIO module uses a stereo camera and IMU to yield low-drift 3D position and yaw orientation and drift-free pitch and roll orientation of the robot base link in t…
University of Notre Dame（圣母大学）；Robotics at Google（谷歌机器人部门）
Specialized motions such as jumping are often achieved on quadruped robots by solving a trajectory optimization problem once and executing the trajectory using a tracking controller. This approach is in parallel with Model Predictive Control (MPC) strategies that commonly control regular gaits via online re-planning. In this work, we present a nonlinear MPC (NMPC) technique that unlocks on-the-fly re-planning of specialized motion skills and regular locomotion within a unified framework. The NMPC reasons about a hybrid kinodynamic model, and is solved using a variant of a constrained Different…
Design and Control of the "QuadBoat": A Quadruped Surface Vehicle for Drowning Rescue
Shenzhen Institute of Artificial Intelligence and Robotics for Society (AIRS)（深圳社会人工智能与机器人研究所）；The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen)（香港中文大学（深圳））；Stevens Institute of Technology（史蒂文斯理工学院）
