How does the structure embedded in learning policy affect learning quadruped locomotion?
Southern University of Science and Technology（南方科技大学）；Massachusetts Institute of Technology（麻省理工学院）；University of British Columbia（不列颠哥伦比亚大学）；National University of Singapore（新加坡国立大学）
Kuangen Zhang, Jongwoo Lee, Zhimin Hou, Clarence W. de Silva, Chenglong Fu, Neville Hogan
Reinforcement learning (RL) is a popular data-driven method that has demonstrated great success in robotics. Previous works usually focus on learning an end-to-end (direct) policy to directly output joint torques. While the direct policy seems convenient, the resultant performance may not meet our expectations. To improve its performance, more sophisticated reward functions or more structured policies can be utilized. This paper focuses on the latter because the structured policy is more intuitive and can inherit insights from previous model-based controllers. It is unsurprising that the structure, such as a better choice of the action space and constraints of motion trajectory, may benefit the training process and the final performance of the policy at the cost of generality, but the quantitative effect is still unclear. To analyze the effect of the structure quantitatively, this paper investigates three policies with different levels of structure in learning quadruped locomotion: a direct policy, a structured policy, and a highly structured policy. The structured policy is trained to learn a task-space impedance controller and the highly structured policy learns a controller tailored for trot running, which we adopt from previous work. To evaluate trained policies, we design a simulation experiment to track different desired velocities under force disturbances. Simulation results show that structured policy and highly structured policy require 1/3 and 3/4 fewer training steps than the direct policy to achieve a similar level of cumulative reward, and seem more robust and efficient than the direct policy. We highlight that the structure embedded in the policies significantly affects the overall performance of learning a complicated task when complex dynamics are involved, such as legged locomotion.
Gait in Eight: Efficient On-Robot Learning for Omnidirectional Quadruped Locomotion
Department of Computer Science, Technical University of Darmstadt（德累斯顿技术大学计算机科学系）；MAB Robtics；German Research Center for AI (DFKI), Research Department: Systems AI for Robot Learning（德国人工智能研究中心（DFKI）系统AI机器人学习研究部）；Robotics Institute Germany (RIG)（德国机器人研究所）
MASQ: Multi-Agent Reinforcement Learning for Single Quadruped Robot Locomotion
Harbin Institute of Technology Shenzhen（哈尔滨工业大学（深圳））；School of Mechanical Engineering and Automation（机械工程与自动化学院）；Guangdong Key Laboratory of Intelligent Morphing Mechanisms and Adaptive Robotics（广东省智能变胞机构与自适应机器人重点实验室）
Dynamic Quadrupedal Legged and Aerial Locomotion via Structure Repurposing
Department of Electrical and Computer Engineering, Northeastern University（电气与计算机工程系，东北大学）；Department of Aerospace Engineering, California Institute of Technology（航空航天工程系，加州理工学院）
Unified Locomotion Transformer with Simultaneous Sim-to-Real Transfer for Quadrupeds
NVIDIA AI Technology Centre (NVAITC)（NVIDIA AI技术中心）；College of Computing and Data Science, Nanyang Technological University（计算与数据科学学院，南洋理工大学）；Nanyang Technological University（南洋理工大学）
Learning Quadrupedal Locomotion over Challenging Terrain
Robotic Systems Lab, ETH Zurich（苏黎世联邦理工学院机器人系统实验室）；Robotics and Artificial Intelligence Lab, KAIST（韩国科学技术院机器人与人工智能实验室）；Intelligent Systems Lab, Intel（英特尔智能系统实验室）
Towards An Adaptive Locomotion Strategy For Quadruped Rovers: Quantifying When To Slide Or Walk On Planetary Slopes
Dynamic Legged Systems Lab, Istituto Italiano di Tecnologia (IIT)（动态腿系统实验室，意大利技术研究院）；Dipartimento di Ingegneria Elettrica Elettronica e Informatica, Università di Catania（电气电子与信息工程系，卡塔尼亚大学）
Physics-informed Neural Network Predictive Control for Quadruped Locomotion
School of Intelligent Systems Engineering, Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区智能系统工程学院）；General Embodied AI Center of Sun Yat-sen University（中山大学通用具身人工智能中心）；Guangdong Provincial Key Laboratory of Fire Science and Intelligent Emergency Technology（广东省火灾科学与智能应急技术重点实验室）；College of Engineering, Peking University（北京大学工学院）；Robotics and Autonomous Systems Thrust, The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）机器人与自主系统部）；George W. Woodruff School of Mechanical Engineering, Georgia Institute of Technology（佐治亚理工学院乔治·W. 伍德拉夫机械工程学院）
Momentum-Aware Trajectory Optimization and Control for Agile Quadrupedal Locomotion
Institute for Robotics and Intelligent Machines, Georgia Institute of Technology（佐治亚理工学院机器人与智能机器研究所）；MIT Biomimetic Robotics Lab（麻省理工学院仿生机器人实验室）；NAVER LABS（NAVER实验室）
End-to-End Multi-Task Policy Learning from NMPC for Quadruped Locomotion
Institute for Artificial Intelligence and Autonomous Systems (A2S)（人工智能与自主系统研究所）；Bonn-Rhein-Sieg University of Applied Sciences（波恩-莱茵-锡格应用科学大学）；Humanoid Robots Lab（人形机器人实验室）；University of Bonn（波恩大学）；Lamarr Institute for Machine Learning and Artificial Intelligence（Lamarr机器学习与人工智能研究所）；Center for Robotics（机器人中心）；Fraunhofer Institute for Intelligent Analysis and Information Systems（弗劳恩霍夫智能分析与信息系统研究所）
