Reinforcement Learning (RL) has seen many recent successes for quadruped robot control. The imitation of reference motions provides a simple and powerful prior for guiding solutions towards desired solutions without the need for meticulous reward design. While much work uses motion capture data or hand-crafted trajectories as the reference motion, relatively little work has explored the use of reference motions coming from model-based trajectory optimization. In this work, we investigate several design considerations that arise with such a framework, as demonstrated through four dynamic behaviours: trot, front hop, 180 backflip, and biped stepping. These are trained in simulation and transferred to a physical Solo 8 quadruped robot without further adaptation. In particular, we explore the space of feed-forward designs afforded by the trajectory optimizer to understand its impact on RL learning efficiency and sim-to-real transfer. These findings contribute to the long standing goal of producing robot controllers that combine the interpretability and precision of model-based optimization with the robustness that model-free RL-based controllers offer.
Unified Locomotion Transformer with Simultaneous Sim-to-Real Transfer for Quadrupeds
NVIDIA AI Technology Centre (NVAITC)(NVIDIA AI技术中心);College of Computing and Data Science, Nanyang Technological University(计算与数据科学学院,南洋理工大学);Nanyang Technological University(南洋理工大学)
Imitation and Adaptation Based on Consistency: A Quadruped Robot Imitates Animals from Videos Using Deep Reinforcement Learning
Shenyang Institute of Automation, Chinese Academy of Sciences(中国科学院沈阳自动化研究所);Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences(中国科学院机器人与智能制造创新研究院);Westlake University(西湖大学);University of Chinese Academy of Sciences(中国科学院大学)
Continuous Control of Diverse Skills in Quadruped Robots Without Complete Expert Datasets
Academy for Engineering and Technology, Fudan University(复旦大学工程与技术学院);Beijing Jingcheng Zhitong Robotics Technology Co.(北京京城智通机器人科技有限公司);Engineering Research Center of AI and Robotics(AI与机器人工程研究中心)
Fine-Tuning Hard-to-Simulate Objectives for Quadruped Locomotion: A Case Study on Total Power Saving
Institute for Interdisciplinary Information Sciences, Tsinghua University(清华大学交叉信息研究院);Shanghai AI Lab(上海人工智能实验室);Shanghai Qi Zhi Institute(上海期智研究院);Department of Electronic Engineering, Tsinghua University(清华大学电子工程系)
Learning Diverse Natural Behaviors for Enhancing the Agility of Quadrupedal Robots
Department of Control Science and Intelligent Engineering(控制科学与智能工程系);School of Management and Engineering(管理与工程学院);Australian Artificial Intelligence Institute(澳大利亚人工智能研究所)
Latent Conditioned Loco-Manipulation Using Motion Priors
LAAS-CNRS, Université de Toulouse(CNRS拉法耶特研究所、图卢兹大学);Artificial and Natural Intelligence Toulouse Institute(图卢兹人工智能与自然智能研究所);University of Edinburgh, School of Informatics(爱丁堡大学信息学院)
Explosive Jumping with Rigid and Articulated Soft Quadrupeds via Example Guided Reinforcement Learning
Cognitive Robotics, Delft University of Technology(德鲁夫特理工大学认知机器人实验室);Department of Computer Science, University of Manchester(曼彻斯特大学计算机科学系);Department of Industrial Engineering, University of Trento(特伦托大学工业工程系);Institute of Robotics and Mechatronics, German Aerospace Center (DLR)(德国航空航天中心机器人与机电一体化研究所)
Safety-Aware Imitation Learning via MPC-Guided Disturbance Injection
Department of Electrical Engineering, Tsinghua University(电子工程系,清华大学);Department of Electrical and Computer Engineering, University of Southern California(电气与计算机工程系,南加州大学);Department of Aeronautics and Astronautics, Stanford University(航空与航天工程系,斯坦福大学)
Motion Priors Reimagined: Adapting Flat-Terrain Skills for Complex Quadruped Mobility
Department of Mechanical Engineering, EPFL(瑞士联邦理工学院机械工程系);ETH AI Center, ETH Zurich(苏黎世联邦理工学院人工智能中心);Robotic Systems Lab, ETH Zurich(苏黎世联邦理工学院机器人系统实验室)
Diverse Skill Discovery for Quadruped Robots via Unsupervised Learning
College of Intelligent Robotics and Advanced Manufacturing(智能机器人与先进制造学院);Fudan University(复旦大学);College of Foreign Language(外国语言学院);The University of Shanghai for Science and Technology(上海科技大学)
Walk Like Dogs: Learning Steerable Imitation Controllers for Legged Robots from Unlabeled Motion Data
Computational Robotics Lab in the Department of Computer Science, ETH Zurich, Switzerland(计算机科学系计算机器人实验室,瑞士苏黎世联邦理工学院);Robot Intelligence Lab in the Department of Artificial Intelligence, Korea University, South Korea(人工智能系机器人智能实验室,韩国大学)
Spatio-Temporal Motion Retargeting for Quadruped Robots
Department of Artificial Intelligence, Korea University(人工智能系,韩国大学);Department of Computer Science, ETH Zurich(计算机科学系,苏黎世联邦理工学院);Department of Mechanical and Aerospace Engineering, UCLA(机械与航空航天工程系,加州大学洛杉矶分校)
Online Trajectory Optimization for Dynamic Aerial Motions of a Quadruped Robot
Massachusetts Institute of Technology(麻省理工学院)
这项工作提出了一个两部分的框架,在线规划和执行的动态空中运动的四足机器人。运动规划通过质心动量为基础的非线性优化,是一般足以产生丰富的新的动态运动集的基础上,仅根据用户指定的接触时间表和所需的发射速度的机器人。由于这种非线性优化对于实时滚动时域控制来说是不容易处理的,因此在空中运动的准备中经由非线性优化来规划运动一次,然后使用基于变分的最优控制器来连续跟踪运动,该基于变分的最优控制器对存在于实际硬件中的不确定性(诸如建模误差或干扰)提供鲁棒性。运动规划通常需要0.05-0.15秒,而最佳控制器在500 Hz时找到稳定的反馈输入。在MIT Mini Cheetah上的实验结果表明,该框架可以可靠地产生成功的空中运动,如跳上和跳下平台、旋转、翻转、桶形滚转和跨越障碍物的跳跃。
Minimizing Acoustic Noise: Enhancing Quiet Locomotion for Quadruped Robots in Indoor Applications
Department of Computer Science and Engineering, Shanghai Jiao Tong University(计算机科学与工程系,上海交通大学);Department of Automation, Shanghai Jiao Tong University(自动化系,上海交通大学);MoE Key Lab of Artificial Intelligence and AI Institute, Shanghai Jiao Tong University(人工智能与AI研究所,上海交通大学)