Learning various motor skills for quadrupedal robots is a challenging problem that requires careful design of task-specific mathematical models or reward descriptions. In this work, we propose to learn a single capable policy using deep reinforcement learning by imitating a large number of reference motions, including walking, turning, pacing, jumping, sitting, and lying. On top of the existing motion imitation framework, we first carefully design the observation space, the action space, and the reward function to improve the scalability of the learning as well as the robustness of the final policy. In addition, we adopt a novel adaptive motion sampling (AMS) method, which maintains a balance between successful and unsuccessful behaviors. This technique allows the learning algorithm to focus on challenging motor skills and avoid catastrophic forgetting. We demonstrate that the learned policy can exhibit diverse behaviors in simulation by successfully tracking both the training dataset and out-of-distribution trajectories. We also validate the importance of the proposed learning formulation and the adaptive motion sampling scheme by conducting experiments.
Robots operating in human environments need various skills, like slow and fast walking, turning, side-stepping, and many more. However, building robot controllers that can exhibit such a large range of behaviors is a challenging problem that requires tedious investigation for every task. We present a unified model-based control algorithm for imitating different animal gaits without expensive simulation training or real-world fine-tuning. Our method consists of stance and swing leg controllers using a centroidal dynamics model augmented with online adaptation techniques. We also develop a whole…
Seoul National University（首尔国立大学）；Georgia Institute of Technology（佐治亚理工学院）
A motion-based control interface promises flexible robot operations in dangerous environments by combining user intuitions with the robot's motor capabilities. However, designing a motion interface for non-humanoid robots, such as quadrupeds or hexapods, is not straightforward because different dynamics and control strategies govern their movements. We propose a novel motion control system that allows a human user to operate various motor tasks seamlessly on a quadrupedal robot. We first retarget the captured human motion into the corresponding robot motion with proper semantics using supervis…
We present a learning algorithm for training a single policy that imitates multiple gaits of a walking robot. To achieve this, we use and extend MPC-Net, which is an Imitation Learning approach guided by Model Predictive Control (MPC). The strategy of MPC-Net differs from many other approaches since its objective is to minimize the control Hamiltonian, which derives from the principle of optimality. To represent the policies, we employ a mixture-of-experts network (MEN) and observe that the performance of a policy improves if each expert of a MEN specializes in controlling exactly one mode of …
Learning Diverse Natural Behaviors for Enhancing the Agility of Quadrupedal Robots
Department of Control Science and Intelligent Engineering（控制科学与智能工程系）；School of Management and Engineering（管理与工程学院）；Australian Artificial Intelligence Institute（澳大利亚人工智能研究所）
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Vanderbilt University（范德堡大学）
We tackle the problem of perceptive locomotion in dynamic environments. In this problem, a quadrupedal robot must exhibit robust and agile walking behaviors in response to environmental clutter and moving obstacles. We present a hierarchical learning framework, named PRELUDE, which decomposes the problem of perceptive locomotion into high-level decision-making to predict navigation commands and low-level gait generation to realize the target commands. In this framework, we train the high-level navigation controller with imitation learning on human demonstrations collected on a steerable cart a…
Walk Like Dogs: Learning Steerable Imitation Controllers for Legged Robots from Unlabeled Motion Data
Computational Robotics Lab in the Department of Computer Science, ETH Zurich, Switzerland（计算机科学系计算机器人实验室，瑞士苏黎世联邦理工学院）；Robot Intelligence Lab in the Department of Artificial Intelligence, Korea University, South Korea（人工智能系机器人智能实验室，韩国大学）
We present a deep reinforcement learning (deep RL) algorithm that consists of learning-based motion planning and imitation to tackle challenging control problems. Deep RL has been an effective tool for solving many high-dimensional continuous control problems, but it cannot effectively solve challenging problems with certain properties, such as sparse reward functions or sensitive dynamics. In this work, we propose an approach that decomposes the given problem into two deep RL stages: motion planning and motion imitation. The motion planning stage seeks to compute a feasible motion plan by lev…
Latent Conditioned Loco-Manipulation Using Motion Priors
LAAS-CNRS, Université de Toulouse（CNRS拉法耶特研究所、图卢兹大学）；Artificial and Natural Intelligence Toulouse Institute（图卢兹人工智能与自然智能研究所）；University of Edinburgh, School of Informatics（爱丁堡大学信息学院）
UC Berkeley（加州大学伯克利分校）；Robotics at Google（谷歌机器人）；Stanford（斯坦福大学）
While robot learning has demonstrated promising results for enabling robots to automatically acquire new skills, a critical challenge in deploying learning-based systems is scale: acquiring enough data for the robot to effectively generalize broadly. Imitation learning, in particular, has remained a stable and powerful approach for robot learning, but critically relies on expert operators for data collection. In this work, we target this challenge, aiming to build an imitation learning system that can continuously improve through autonomous data collection, while simultaneously avoiding the ex…
Safety-Aware Imitation Learning via MPC-Guided Disturbance Injection
Department of Electrical Engineering, Tsinghua University（电子工程系，清华大学）；Department of Electrical and Computer Engineering, University of Southern California（电气与计算机工程系，南加州大学）；Department of Aeronautics and Astronautics, Stanford University（航空与航天工程系，斯坦福大学）
Columbia University（哥伦比亚大学）；Meta AI；Carnegie Mellon University（卡内基梅隆大学）
Large-scale training have propelled significant progress in various sub-fields of AI such as computer vision and natural language processing. However, building robot learning systems at a comparable scale remains challenging. To develop robots that can perform a wide range of skills and adapt to new scenarios, efficient methods for collecting vast and diverse amounts of data on physical robot systems are required, as well as the capability to train high-capacity policies using such datasets. In this work, we propose a framework for scaling robot learning, with specific focus on multi-task and …
Beyond Imitation: Reinforcement Learning-Based Sim-Real Co-Training for VLA Models
Tsinghua University（清华大学）；Harbin Institute of Technology（哈尔滨工业大学）；Peking University（北京大学）；Carnegie Mellon University（卡内基梅隆大学）；Shanghai AI Laboratory（上海人工智能实验室）；Zhongguancun Academy（中关村学院）
Robotics at Google（谷歌机器人部门）；X, The Moonshot Factory（X（登月工厂））；UC Berkeley（加州大学伯克利分校）
Robotic skills can be learned via imitation learning (IL) using user-provided demonstrations, or via reinforcement learning (RL) using large amountsof autonomously collected experience.Both methods have complementarystrengths and weaknesses: RL can reach a high level of performance, but requiresexploration, which can be very time consuming and unsafe; IL does not requireexploration, but only learns skills that are as good as the provided demonstrations.Can a single method combine the strengths of both approaches? A number ofprior methods have aimed to address this question, proposing a variety…
Scalable Multi Agent Diffusion Policies for Coverage Control
Department of Electrical and Systems Engineering, University of Pennsylvania（宾夕法尼亚大学电气与系统工程系）；Department of Mechanical Engineering, Indian Institute of Technology Bombay (IITB)（印度理工学院班加罗尔机械工程系）
Translating Flow to Policy via Hindsight Online Imitation
Institute for Interdisciplinary Information Sciences, Tsinghua University（清华大学交叉信息研究院）；University of California San Diego（加州大学圣地亚哥分校）；Washington University in St. Louis（圣路易斯华盛顿大学）；Shanghai Jiao Tong University（上海交通大学）；Shanghai Qi Zhi Institute（上海启智研究所）
While imitation learning provides us with an efficient toolkit to train robots, learning skills that are robust to environment variations remains a significant challenge. Current approaches address this challenge by relying either on large amounts of demonstrations that span environment variations or on handcrafted reward functions that require state estimates. Both directions are not scalable to fast imitation. In this work, we present Fast Imitation of Skills from Humans (FISH), a new imitation learning approach that can learn robust visual skills with less than a minute of human demonstrati…
Learning a Unified Policy for Position and Force Control in Legged Loco-Manipulation
State Key Laboratory of General Artificial Intelligence, BIGAI（一般人工智能国家重点实验室，BIGAI）；Joint Laboratory of Embodied AI and Humanoid Robots, BIGAI & UniTree Robotics（具身人工智能与仿人机器人联合实验室，BIGAI与UniTree机器人）；Beijing University of Posts and Telecommunications（北京邮电大学）
California Institute of Technology（加州理工学院）；Virginia Polytechnic Institute and State University（弗吉尼亚理工学院暨州立大学）
The hybrid zero dynamics (HZD) approach has become a powerful tool for the gait planning and control of bipedal robots. This paper aims to extend the HZD methods to address walking, ambling and trotting behaviors on a quadrupedal robot. We present a framework that systematically generates a wide range of optimal trajectories and then provably stabilizes them for the full-order, nonlinear and hybrid dynamical models of quadrupedal locomotion. The gait planning is addressed through a scalable nonlinear programming using direct collocation and HZD. The controller synthesis for the exponential sta…
GRAIL: Goal Recognition Alignment through Imitation Learning
Department of Computer Science, Bar Ilan University（巴伊兰大学计算机科学系）；Department of Computer Science, Aberdeen University（阿伯丁大学计算机科学系）；Department of Computer Science, Tufts University（塔夫茨大学计算机科学系）
