arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1909.05557cs.LGcs.AIstat.ML
收藏

带收缩的模块化元学习

Modular Meta-Learning with Shrinkage

DeepMind(深度思维)

Yutian Chen, Abram L. Friesen, Feryal Behbahani, Arnaud Doucet, David Budden, Matthew W. Hoffman, Nando de Freitas

摘要:

许多现实世界的问题,包括多说话者文本到语音合成,可以大大受益于元学习大型模型的能力,只有几个特定于任务的组件。只更新这些特定于任务的模块,然后允许模型适应低数据任务,并根据需要进行尽可能多的步骤,而不会冒过拟合的风险。不幸的是,现有的元学习方法要么不能扩展到长期适应,要么依赖于手工制作的特定于任务的架构。在这里,我们提出了一个元学习的方法,避免了这种往往次优的手选择的需要。特别是,我们开发了基于贝叶斯收缩的通用技术,以自动发现和学习特定任务和通用可重用模块。从经验上讲,我们证明我们的方法发现了一小部分有意义的特定于任务的模块,并且在任务数据很少和适应范围较长的少量文本到语音(Few-Shot text to speech)等领域中优于现有的元学习方法。我们还表明,现有的元学习方法,包括MAML,iMAML和爬行动物出现我们的方法的特殊情况。

英文摘要:

Many real-world problems, including multi-speaker text-to-speech synthesis, can greatly benefit from the ability to meta-learn large models with only a few task-specific components. Updating only these task-specific modules then allows the model to be adapted to low-data tasks for as many steps as necessary without risking overfitting. Unfortunately, existing meta-learning methods either do not scale to long adaptation or else rely on handcrafted task-specific architectures. Here, we propose a meta-learning approach that obviates the need for this often sub-optimal hand-selection. In particular, we develop general techniques based on Bayesian shrinkage to automatically discover and learn both task-specific and general reusable modules. Empirically, we demonstrate that our method discovers a small set of meaningful task-specific modules and outperforms existing meta-learning approaches in domains like few-shot text-to-speech that have little task data and long adaptation horizons. We also show that existing meta-learning methods including MAML, iMAML, and Reptile emerge as special cases of our method.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑