arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1611.06194cs.CVcs.AIstat.ML
收藏

专家门:通过专家网络终身学习

Expert Gate: Lifelong Learning with a Network of Experts

KU Leuven(鲁汶大学);IMEC(比利时微电子研究中心)

Rahaf Aljundi, Punarjay Chakravarty, Tinne Tuytelaars

摘要:

在本文中,我们介绍了一个终身学习的模型,基于网络的专家。新的任务/专家被学习并顺序地添加到模型中,建立在之前学习的基础上。为了确保该过程的可扩展性,来自先前任务的数据不能被存储,因此在学习新任务时不可用。在这种情况下,一个关键的问题,没有在文献中提到,到目前为止,涉及到的决定,专家部署在测试时。我们引入了一组门控自动编码器,它可以学习手头任务的表示,并在测试时自动将测试样本转发给相关专家。这也带来了内存效率,因为在任何给定时间只有一个专家网络必须加载到内存中。此外,自动编码器固有地捕获一个任务与另一个任务的相关性,基于该相关性,可以选择用于训练新专家的最相关的先验模型,并进行微调或学习而不会忘记。我们评估我们的方法对图像分类和视频预测问题。

英文摘要:

In this paper we introduce a model of lifelong learning, based on a Network of Experts. New tasks / experts are learned and added to the model sequentially, building on what was learned before. To ensure scalability of this process,data from previous tasks cannot be stored and hence is not available when learning a new task. A critical issue in such context, not addressed in the literature so far, relates to the decision which expert to deploy at test time. We introduce a set of gating autoencoders that learn a representation for the task at hand, and, at test time, automatically forward the test sample to the relevant expert. This also brings memory efficiency as only one expert network has to be loaded into memory at any given time. Further, the autoencoders inherently capture the relatedness of one task to another, based on which the most relevant prior model to be used for training a new expert, with finetuning or learning without-forgetting, can be selected. We evaluate our method on image classification and video prediction problems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑