We propose HyperDynamics, a dynamics meta-learning framework that conditions on an agent's interactions with the environment and optionally its visual observations, and generates the parameters of neural dynamics models based on inferred properties of the dynamical system. Physical and visual properties of the environment that are not part of the low-dimensional state yet affect its temporal dynamics are inferred from the interaction history and visual observations, and are implicitly captured in the generated parameters. We test HyperDynamics on a set of object pushing and locomotion tasks. It outperforms existing dynamics models in the literature that adapt to environment variations by learning dynamics over high dimensional visual observations, capturing the interactions of the agent in recurrent state representations, or using gradient-based meta-optimization. We also show our method matches the performance of an ensemble of separately trained experts, while also being able to generalize well to unseen environment variations at test time. We attribute its good performance to the multiplicative interactions between the inferred system properties -- captured in the generated parameters -- and the low-dimensional state representation of the dynamical system.
ETH AI Center, ETH Zürich（苏黎世联邦理工学院ETH人工智能中心）；Institute of Neuroinformatics, University of Zürich and ETH Zürich（苏黎世大学与苏黎世联邦理工学院神经信息学研究所）；Soft Robotics Lab, ETH Zürich（苏黎世联邦理工学院软机器人实验室）；Dept. of Computer Science, ETH Zürich（苏黎世联邦理工学院计算机科学系）
We introduce meta-learning algorithms that perform zero-shot weight-space adaptation of neural network models to unseen tasks. Our methods repurpose the popular generative image synthesis techniques of natural language guidance and diffusion models to generate neural network weights adapted for tasks. We first train an unconditional generative hypernetwork model to produce neural network weights; then we train a second "guidance" model that, given a natural language task description, traverses the hypernetwork latent space to find high-performance task-adapted weights in a zero-shot manner. We…
HyperMAML: Few-Shot Adaptation of Deep Models with Hypernetworks
Jagiellonian University（雅盖隆大学）；Doctoral School of Exact and Natural Sciences（精确与自然科学博士学院）；IDEAS NCBR；Tooploox Ltd.（Tooploox有限公司）；Wroclaw University of Science and Technology（弗罗茨瓦夫理工大学）
针对 MAML 依赖梯度更新、适配步少则不足、步多则难训练且易过拟合的问题，本文提出 HyperMAML，用可训练超网络生成权重更新，并在标准少样本基准上取得优于 MAML、比肩 SOTA 的效果。
Microsoft Research Cambridge（微软研究院剑桥分院）；University of Cambridge（剑桥大学）；University of Vienna（维也纳大学）；Eedi
While deep learning has obtained state-of-the-art results in many applications, the adaptation of neural network architectures to incorporate new output features remains a challenge, as neural networks are commonly trained to produce a fixed output dimension. This issue is particularly severe in online learning settings, where new output features, such as items in a recommender system, are added continually with few or no associated observations. As such, methods for adapting neural networks to novel features which are both time and data-efficient are desired. To address this, we propose the C…
Institute of Neuroinformatics（神经信息学研究所）；University of Zürich（苏黎世大学）；ETH Zürich（苏黎世联邦理工学院）
Learning a sequence of tasks without access to i.i.d. observations is a widely studied form of continual learning (CL) that remains challenging. In principle, Bayesian learning directly applies to this setting, since recursive and one-off Bayesian updates yield the same result. In practice, however, recursive updating often leads to poor trade-off solutions across tasks because approximate inference is necessary for most models of interest. Here, we describe an alternative Bayesian approach where task-conditioned parameter distributions are continually inferred from data. We offer a practical …
Training a reinforcement learning (RL) agent on a real-world robotics task remains generally impractical due to sample inefficiency. Multi-task RL and meta-RL aim to improve sample efficiency by generalizing over a distribution of related tasks. However, doing so is difficult in practice: In multi-task RL, state of the art methods often fail to outperform a degenerate solution that simply learns each task separately. Hypernetworks are a promising path forward since they replicate the separate policies of the degenerate solution while also allowing for generalization across tasks, and are appli…
University of Oxford（牛津大学）；University College London（伦敦大学学院）
When experience is scarce, models may have insufficient information to adapt to a new task. In this case, auxiliary information - such as a textual description of the task - can enable improved task inference and adaptation. In this work, we propose an extension to the Model-Agnostic Meta-Learning algorithm (MAML), which allows the model to adapt using auxiliary information as well as task experience. Our method, Fusion by Meta-Initialization (FuMI), conditions the model initialization on auxiliary information using a hypernetwork, rather than learning a single, task-agnostic initialization. F…
Novateur Research Solutions（诺瓦图尔研究解决方案公司）；UC San Diego（加州大学圣地亚哥分校）
A need to understand and predict vehicles' behavior underlies both public and private goals in the transportation domain, including urban planning and management, ride-sharing services, and intelligent transportation systems. Individuals' preferences and intended destinations vary throughout the day, week, and year: for example, bars are most popular in the evenings, and beaches are most popular in the summer. Despite this principle, we note that recent studies on a popular benchmark dataset from Porto, Portugal have found, at best, only marginal improvements in predictive performance from inc…
Meta-Transfer Learning Powered Temporal Graph Networks for Cross-City Real Estate Appraisal
The Hong Kong University of Science and Technology（香港科技大学）；Shandong University（山东大学）；Computer Network Information Center, Chinese Academy of Sciences（中国科学院计算机网络信息中心）
Beijing International Center for Mathematical Research, Peking University（北京大学北京国际数学研究中心）；Beijing Engineering Research Center of Radiographic Techniques and Equipment, Institute of High Energy Physics, Chinese Academy of Sciences（中国科学院高能物理研究所北京市射线成像技术与装备工程研究中心）；School of Nuclear Science and Technology, University of Chinese Academy of Sciences（中国科学院大学核科学与技术学院）；Department of Electrical and Computer Engineering, University of Massachusetts Lowell（马萨诸塞大学洛厄尔分校电气与计算机工程系）
X-ray Computed Tomography (CT) is widely used in clinical applications such as diagnosis and image-guided interventions. In this paper, we propose a new deep learning based model for CT image reconstruction with the backbone network architecture built by unrolling an iterative algorithm. However, unlike the existing strategy to include as many data-adaptive components in the unrolled dynamics model as possible, we find that it is enough to only learn the parts where traditional designs mostly rely on intuitions and experience. More specifically, we propose to learn an initializer for the conju…
