arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1908.10223cs.LGstat.ML
收藏

从低拍数据流中持续学习

Learning Continually from Low-shot Data Stream

Xiamen University(厦门大学);DAMO Academy, Alibaba Group(阿里巴巴集团达摩院)

Canyu Le, Xihan Wei, Biao Wang, Lei Zhang, Zhonggui Chen

摘要:

虽然深度学习在各种应用中取得了显着的成果,但它通常是数据饥饿的,并且难以在非平稳数据流上学习。为了解决这两个限制,深度学习模型不仅要能够从少量数据中学习,还要随着时间的推移从数据流中增量学习新概念,而不会忘记以前的知识。有限的文献同时解决这两个问题。在这项工作中,我们提出了一种新的方法,MetaCL,它使神经网络能够有效地学习Meta知识,从低拍摄数据流没有灾难性遗忘。MetaCL训练模型以利用数据的内在特征(即Meta知识),并动态惩罚重要的模型参数变化以保留学习到的知识。通过这种方式,深度学习模型可以有效地从少量数据中获取新知识,并且仍然保持对先前任务的高性能。MetaCL概念简单,易于实现,与模型无关。我们实现了我们的方法在最近的三个正则化为基础的方法。大量的实验表明,我们的方法导致国家的最先进的性能在图像分类基准。

英文摘要:

While deep learning has achieved remarkable results on various applications, it is usually data hungry and struggles to learn over non-stationary data stream. To solve these two limits, the deep learning model should not only be able to learn from a few of data, but also incrementally learn new concepts from data stream over time without forgetting the previous knowledge. Limited literature simultaneously address both problems. In this work, we propose a novel approach, MetaCL, which enables neural networks to effectively learn meta knowledge from low-shot data stream without catastrophic forgetting. MetaCL trains a model to exploit the intrinsic feature of data (i.e. meta knowledge) and dynamically penalize the important model parameters change to preserve learned knowledge. In this way, the deep learning model can efficiently obtain new knowledge from small volume of data and still keep high performance on previous tasks. MetaCL is conceptually simple, easy to implement and model-agnostic. We implement our method on three recent regularization-based methods. Extensive experiments show that our approach leads to state-of-the-art performance on image classification benchmarks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑