Learning adaptable policies is crucial for robots to operate autonomously in our complex and quickly changing world. In this work, we present a new meta-learning method that allows robots to quickly adapt to changes in dynamics. In contrast to gradient-based meta-learning algorithms that rely on second-order gradient estimation, we introduce a more noise-tolerant Batch Hill-Climbing adaptation operator and combine it with meta-learning based on evolutionary strategies. Our method significantly improves adaptation to changes in dynamics in high noise settings, which are common in robotics applications. We validate our approach on a quadruped robot that learns to walk while subject to changes in dynamics. We observe that our method significantly outperforms prior gradient-based approaches, enabling the robot to adapt its policy to changes based on less than 3 minutes of real data.
The process of robot design is a complex task and the majority of design decisions are still based on human intuition or tedious manual tuning. A more informed way of facing this task is computational design methods where design parameters are concurrently optimized with corresponding controllers. Existing approaches, however, are strongly influenced by predefined control rules or motion templates and cannot provide end-to-end solutions. In this paper, we present a design optimization framework using model-free meta reinforcement learning, and its application to the optimizing kinematics and a…
MetaFollower: Adaptable Personalized Autonomous Car Following
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；Tongji University（同济大学）；University of Washington（华盛顿大学）
Learning controllers that reproduce legged locomotion in nature has been a long-time goal in robotics and computer graphics. While yielding promising results, recent approaches are not yet flexible enough to be applicable to legged systems of different morphologies. This is partly because they often rely on precise motion capture references or elaborate learning environments that ensure the naturality of the emergent locomotion gaits but prevent generalization. This work proposes a generic approach for ensuring realism in locomotion by guiding the learning process with the spring-loaded invert…
Georgia Institute of Technology（佐治亚理工学院）；Robotics at Google（谷歌机器人部门）
The ability to walk in new scenarios is a key milestone on the path toward real-world applications of legged robots. In this work, we introduce Meta Strategy Optimization, a meta-learning algorithm for training policies with latent variable inputs that can quickly adapt to new scenarios with a handful of trials in the target environment. The key idea behind MSO is to expose the same adaptation process, Strategy Optimization (SO), to both the training and testing phases. This allows MSO to effectively learn locomotion skills as well as a latent space that is suitable for fast adaptation. We eva…
If robots are to become ubiquitous, they will need to be able to adapt to complex and dynamic environments. Robots that can adapt their bodies while deployed might be flexible and robust enough to meet this challenge. Previous work on dynamic robot morphology has focused on simulation, combining simple modules, or switching between locomotion modes. Here, we present an alternative approach: a self-reconfigurable morphology that allows a single four-legged robot to actively adapt the length of its legs to different environments. We report the design of our robot, as well as the results of a stu…
University of Zürich（苏黎世大学）；ETH Zürich（苏黎世联邦理工学院）；Mila；University of Montreal（蒙特利尔大学）；ServiceNow
Finding neural network weights that generalize well from small datasets is difficult. A promising approach is to learn a weight initialization such that a small number of weight changes results in low generalization error. We show that this form of meta-learning can be improved by letting the learning algorithm decide which weights to change, i.e., by learning where to learn. We find that patterned sparsity emerges from this process, with the pattern of sparsity varying on a problem-by-problem basis. This selective sparsity results in better generalization and less interference in a range of f…
Henry M. Gunn High School（亨利·M·冈恩高中）；OpenAI；UC Berkeley（加州大学伯克利分校）
We develop a metalearning approach for learning hierarchically structured policies, improving sample efficiency on unseen tasks through the use of shared primitives---policies that are executed for large numbers of timesteps. Specifically, a set of primitives are shared within a distribution of tasks, and are switched between by task-specific policies. We provide a concrete metric for measuring the strength of such hierarchies, leading to an optimization problem for quickly reaching high reward on unseen tasks. We then present an algorithm to solve this problem end-to-end through the use of an…
Towards a Dynamic Future with Adaptable Computing and Network Convergence (ACNC)
ICTFicial；Oulu University（奥卢大学）；Ruhr University Bochum（波鸿鲁尔大学）；Southeast University（东南大学）；Seoul National University（首尔大学）；Sejong University（世宗大学）；Futurewei Technologies（未来科技公司（Futurewei Technologies））
Model-based control usually relies on an accurate model, which is often obtained from CAD and actuator models. The more accurate the model the better the control performance. However, in bipedal robots that demonstrate high agility actions, such as running and hopping, the robot hardware will suffer from impacts with the environment and deform in vulnerable parts, which invalidates the predefined model. Thus, it is desired to have an adaptable kinematic structure that takes deformation into consideration. To account for this we propose an approach that models all of the robotic joints as 6-DOF…
Shenzhen Institute of Artificial Intelligence and Robotics for Society (AIRS)（深圳市人工智能与机器人研究院）；Institute of Robotics and Intelligent Manufacturing (IRIM), The Chinese University of Hong Kong (CUHK)（香港中文大学机器人与智能制造研究院）；George W. Woodruff School of Mechanical Engineering, Georgia Institute of Technology（佐治亚理工学院乔治·W·伍德拉夫机械工程学院）；Beijing Institute for General Artificial Intelligence（北京通用人工智能研究院）
Dynamic quadrupedal locomotion over rough terrains reveals remarkable progress over the last few decades. Small-scale quadruped robots are adequately flexible and adaptable to traverse uneven terrains along sagittal direction, such as slopes and stairs. To accomplish autonomous locomotion navigation in complex environments, spinning is a fundamental yet indispensable functionality for legged robots. However, spinning behaviors of quadruped robots on uneven terrain often exhibit position drifts. Motivated by this problem, this study presents an algorithmic method to enable accurate spinning mot…
GenTe: Generative Real-world Terrains for General Legged Robot Locomotion Control
School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）理工学院）；Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳市人工智能与机器人研究院）
We present a meta-learning framework for interactive medical image registration. Our proposed framework comprises three components: a learning-based medical image registration algorithm, a form of user interaction that refines registration at inference, and a meta-learning protocol that learns a rapidly adaptable network initialization. This paper describes a specific algorithm that implements the registration, interaction and meta-learning protocol for our exemplar clinical application: registration of magnetic resonance (MR) imaging to interactively acquired, sparsely-sampled transrectal ult…
Model Agnostic Meta Learning (MAML) is widely used to find a good initialization for a family of tasks. Despite its success, a critical challenge in MAML is to calculate the gradient w.r.t. the initialization of a long training trajectory for the sampled tasks, because the computation graph can rapidly explode and the computational cost is very expensive. To address this problem, we propose Adjoint MAML (A-MAML). We view gradient descent in the inner optimization as the evolution of an Ordinary Differential Equation (ODE). To efficiently compute the gradient of the validation loss w.r.t. the i…
Chandra Family Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, TX, USA（Chandra Family 电子与计算机工程系，德克萨斯大学奥斯汀分校）；Snap, Inc.（Snap公司）；Google Research（Google研究）
Spectral Normalization for Lipschitz-Constrained Policies on Learning Humanoid Locomotion
Department of Intelligence and Information, Graduate School of Convergence Science and Technology, Seoul National University（智能信息系，融合科学与技术研究院，首尔国立大学）；X Technologies（1X技术公司）；Department of Intelligence and Information, Graduate School of Convergence Science and Technology, ASRI, AIIS, Seoul National University, Republic of Korea, and Advanced Institute of Convergence Technology (AICT), Suwon, Republic of Korea（智能信息系，融合科学与技术研究院，ASRI、AIIS、首尔国立大学，韩国，以及融合技术研究所（AICT），Suwon，韩国）
IBM Research AI（IBM研究人工智能中心）；Tel Aviv University（特拉维夫大学）；Technion（以色列理工学院）
Few-Shot Learning (FSL) is a topic of rapidly growing interest. Typically, in FSL a model is trained on a dataset consisting of many small tasks (meta-tasks) and learns to adapt to novel tasks that it will encounter during test time. This is also referred to as meta-learning. Another topic closely related to meta-learning with a lot of interest in the community is Neural Architecture Search (NAS), automatically finding optimal architecture instead of engineering it manually. In this work, we combine these two aspects of meta-learning. So far, meta-learning FSL methods have focused on optimizin…
Human Activity Recognition~(HAR) is the classification of human movement, captured using one or more sensors either as wearables or embedded in the environment~(e.g. depth cameras, pressure mats). State-of-the-art methods of HAR rely on having access to a considerable amount of labelled data to train deep architectures with many train-able parameters. This becomes prohibitive when tasked with creating models that are sensitive to personal nuances in human movement, explicitly present when performing exercises. In addition, it is not possible to collect training data to cover all possible subje…
MLDGG: Meta-Learning for Domain Generalization on Graphs
College of Intelligence and Computing, Tianjin University（天津大学智能与计算学部）；Department of Computer Science, Baylor University（贝勒大学计算机科学系）；School of New Media and Communication, Tianjin University（天津大学新媒体与传播学院）
Oxford Robotics Institute, University of Oxford（牛津大学牛津机器人研究所）
In this paper, we present a factor-graph LiDAR-SLAM system which incorporates a state-of-the-art deeply learned feature-based loop closure detector to enable a legged robot to localize and map in industrial environments. These facilities can be badly lit and comprised of indistinct metallic structures, thus our system uses only LiDAR sensing and was developed to run on the quadruped robot's navigation PC. Point clouds are accumulated using an inertial-kinematic state estimator before being aligned using ICP registration. To close loops we use a loop proposal mechanism which matches individual …
Impressive animal locomotion capabilities are mediated by the co-evolution of the skeletal morphology and muscular properties. Legged robot performance would also likely benefit from the co-optimization of actuators and leg morphology. However, development of custom actuators for legged robots is often expensive and time consuming, which discourages roboticists from pursuing performance gains afforded by application-specific actuator optimization. This paper presents open-source designs for two quasi-direct-drive actuators with performance regimes appropriate for an 8--15 kg robot, built compl…
Tokyo Institute of Technology（东京工业大学）；Honda Research Institute Japan Co., Ltd.（本田日本研究所）
We describe a novel metric-based learning approach that introduces a multimodal framework and uses deep audio and geophone encoders in siamese configuration to design an adaptable and lightweight supervised model. This framework eliminates the need for expensive data labeling procedures and learns general-purpose representations from low multisensory data obtained from omnipresent sensing systems. These sensing systems provide numerous applications and various use cases in activity recognition tasks. Here, we intend to explore the human footstep movements from indoor environments and analyze r…
Context-Based Meta Reinforcement Learning for Robust and Adaptable Peg-in-Hole Assembly Tasks
Humanoid Robots Lab and Center for Robotics, University of Bonn, Germany（波恩大学人形机器人实验室和机器人中心）；Lamarr Institute for Machine Learning and Artificial Intelligence, Bonn, Germany（波恩大学拉马尔人工智能与机器学习研究所）；Cyber Physical Systems Lab, Egypt Japan University of Science and Technology, Alexandria, Egypt（埃及日本科学技术大学网络物理系统实验室）；Faculty of Engineering, Alexandria University, Alexandria, Egypt（亚历山大大学工程学院）；Mechatronics Department, Ain Shams University, Cairo, Egypt（亚历山大大学机电系）
Although reinforcement learning methods can achieve impressive results in simulation, the real world presents two major challenges: generating samples is exceedingly expensive, and unexpected perturbations or unseen situations cause proficient but specialized policies to fail at test time. Given that it is impractical to train separate policies to accommodate all situations the agent may see in the real world, this work proposes to learn how to quickly and effectively adapt online to new tasks. To enable sample-efficient learning, we consider learning online adaptation in the context of model-…
Multistage decision policies provide useful control strategies in high-dimensional state spaces, particularly in complex control tasks. However, they exhibit weak performance guarantees in the presence of disturbance, model mismatch, or model uncertainties. This brittleness limits their use in high-risk scenarios. We present how to quantify the sensitivity of such policies in order to inform of their robustness capacity. We also propose a minimax iterative dynamic game framework for designing robust policies in the presence of disturbance/uncertainties. We test the quantification hypothesis on…
