Efficient Learning of Control Policies for Robust Quadruped Bounding using Pretrained Neural Networks
Institute of Cyber-Systems and Control, Zhejiang University（浙江大学控制科学与工程学院）；Intelligent Robot Research Center, Zhejiang Lab（之江实验室智能机器人研究中心）；Department of Computer Science, University College London（伦敦大学学院计算机科学系）；State Key Laboratory of Industrial Control Technology, Zhejiang University（浙江大学工业控制技术国家重点实验室）
Bounding is one of the important gaits in quadrupedal locomotion for negotiating obstacles. The authors proposed an effective approach that can learn robust bounding gaits more efficiently despite its large variation in dynamic body movements. The authors first pretrained the neural network (NN) based on data from a robot operated by conventional model based controllers, and then further optimised the pretrained NN via deep reinforcement learning (DRL). In particular, the authors designed a reward function considering contact points and phases to enforce the gait symmetry and periodicity, which improved the bounding performance. The NN based feedback controller was learned in the simulation and directly deployed on the real quadruped robot Jueying Mini successfully. A variety of environments are presented both indoors and outdoors with the authors approach. The authors approach shows efficient computing and good locomotion results by the Jueying Mini quadrupedal robot bounding over uneven terrain.
It is often overlooked by roboticists when designing locomotion controllers for their legged machines, that energy consumption plays an important role in selecting the best gaits for locomotion at high speeds or over long distances. The purpose of this study is to examine four similar asymmetrical quadrupedal gaits that are frequently observed in legged animals in nature. To understand how a specific footfall pattern will change the energetics of a legged system, we first developed a full body model of a quadrupedal robot called A1. And for each gait we created a hybrid system with desired foo…
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；New York University（纽约大学）
We present a general, two-stage reinforcement learning approach to create robust policies that can be deployed on real robots without any additional training using a single demonstration generated by trajectory optimization. The demonstration is used in the first stage as a starting point to facilitate initial exploration. In the second stage, the relevant task reward is optimized directly and a policy robust to environment uncertainties is computed. We demonstrate and examine in detail the performance and robustness of our approach on highly dynamic hopping and bounding tasks on a quadruped r…
Robert Bosch Centre for Cyber-Physical Systems, IISc（印度科学学院罗伯特·博世网络物理系统中心）；BITS Pilani（博拉理工学院皮拉尼校区）
In this work, we provide a simulation framework to perform systematic studies on the effects of spinal joint compliance and actuation on bounding performance of a 16-DOF quadruped spined robot Stoch 2. Fast quadrupedal locomotion with active spine is an extremely hard problem, and involves a complex coordination between the various degrees of freedom. Therefore, past attempts at addressing this problem have not seen much success. Deep-Reinforcement Learning seems to be a promising approach, after its recent success in a variety of robot platforms, and the goal of this paper is to use this appr…
Zhejiang University（浙江大学）；ZJU-Hangzhou Global Scientific and Technological Innovation Center（浙江大学-杭州市全球科技创新中心）；State Key Laboratory of Fluid Power and Mechatronic System, Zhejiang University（浙江大学流体动力与机电系统国家重点实验室）；Institute of Applied Mechanics, Zhejiang University（浙江大学应用力学研究所）
Gaits and transitions are key components in legged locomotion. For legged robots, describing and reproducing gaits as well as transitions remain longstanding challenges. Reinforcement learning has become a powerful tool to formulate controllers for legged robots. Learning multiple gaits and transitions, nevertheless, is related to the multi-task learning problems. In this work, we present a novel framework for training a simple control policy for a quadruped robot to locomote in various gaits. Four independent phases are used as the interface between the gait generator and the control policy, …
Deep reinforcement learning has emerged as a popular and powerful way to develop locomotion controllers for quadruped robots. Common approaches have largely focused on learning actions directly in joint space, or learning to modify and offset foot positions produced by trajectory generators. Both approaches typically require careful reward shaping and training for millions of time steps, and with trajectory generators introduce human bias into the resulting control policies. In this paper, we present a learning framework that leads to the natural emergence of fast and robust bounding policies …
Robert Bosch Centre for Cyber-Physical Systems, IISc, Bangalore, India（印度班加罗尔印度科学学院罗伯特·博世网络物理系统中心）；Faculty of Mechanical Engineering（机械工程学院）；Faculty of Electrical & Computer Engineering（电气与计算机工程学院）；Computer Science & Automation（计算机科学与自动化学院）；Robert Bosch Center for Cyber Physical Systems, IISc, Bengaluru, India（印度班加罗尔印度科学学院罗伯特·博世网络物理系统中心）
In this paper, we present a complete description of the hardware design and control architecture of our custom built quadruped robot, called the `Stoch'. Our goal is to realize a robust, modular, and a reliable quadrupedal platform, using which various locomotion behaviors are explored. This platform enables us to explore different research problems in legged locomotion, which use both traditional and learning based techniques. We discuss the merits and limitations of the platform in terms of exploitation of available behaviours, fast rapid prototyping, reproduction and repair. Towards the end…
Interlimb neural connection is not required for gait transition in quadruped locomotion
Institute of Mathematics for Industry, Kyushu University（工业数学研究所，九州大学）；Research Institute for Electronics Science, Hokkaido University（电子科学研究所，北海道大学）；Research Institute of Electrical Communication, Tohoku University（电气通信研究所，东北大学）；Japan Science and Technology Agency CREST（日本科学技术 Agency CREST）
Preference Aligned Diffusion Planner for Quadrupedal Locomotion Control
Graduate School of Engineering Science, Osaka University（大阪大学工程科学研究院）；Robotics and Autonomous Systems Thrust, The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）机器人与自主系统方向）；Institute of Artificial Intelligence (TeleAI), China Telecom（中国电信人工智能研究院（TeleAI））；Southern University of Science and Technology（南方科技大学）；School of Transportation, Southeast University（东南大学交通学院）
The feet of robots are typically used to design locomotion strategies, such as balancing, walking, and running. However, they also have great potential to perform manipulation tasks. In this paper, we propose a model predictive control (MPC) framework for a quadrupedal robot to dynamically balance on a ball and simultaneously manipulate it to follow various trajectories such as straight lines, sinusoids, circles and in-place turning. We numerically validate our controller on the Mini Cheetah robot using different gaits including trotting, bounding, and pronking on the ball.
University of Notre Dame（圣母大学）；Robotics at Google（谷歌机器人部门）
Specialized motions such as jumping are often achieved on quadruped robots by solving a trajectory optimization problem once and executing the trajectory using a tracking controller. This approach is in parallel with Model Predictive Control (MPC) strategies that commonly control regular gaits via online re-planning. In this work, we present a nonlinear MPC (NMPC) technique that unlocks on-the-fly re-planning of specialized motion skills and regular locomotion within a unified framework. The NMPC reasons about a hybrid kinodynamic model, and is solved using a variant of a constrained Different…
Optimal control is a successful approach to generate motions for complex robots, in particular for legged locomotion. However, these techniques are often too slow to run in real time for model predictive control or one needs to drastically simplify the dynamics model. In this work, we present a method to learn to predict the gradient and hessian of the problem value function, enabling fast resolution of the predictive control problem with a one-step quadratic program. In addition, our method is able to satisfy constraints like friction cones and unilateral constraints, which are important for …
Multi-contact Stochastic Predictive Control for Legged Robots with Contact Locations Uncertainty
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Tandon School of Engineering, New York University（纽约大学坦登工程学院）；Munich Institute of Robotics and Machine Intelligence, Technical University of Munich（慕尼黑工业大学慕尼黑机器人与机器智能研究所）；University of Trento（特伦托大学）
Identifying Important Sensory Feedback for Learning Locomotion Skills
University of Edinburgh（爱丁堡大学）；Shenzhen Amigaga Technology Co Ltd（深圳阿米嘎嘎科技有限公司）；École Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院（EPFL））；University College London（伦敦大学学院）
