arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2104.01678cs.LGcs.AI
收藏

利用数据分布漂移分析了解持续学习设置

Understanding Continual Learning Settings with Data Distribution Drift Analysis

Université de Montréal(蒙特利尔大学);MILA - Quebec AI Institute(MILA - 魁北克人工智能研究所)

Timothée Lesort, Massimo Caccia, Irina Rish

摘要:

经典的机器学习算法通常假设数据是独立同分布绘制的。一个平稳的概率分布。最近,持续学习作为机器学习的一个快速增长的领域出现,其中这个假设是放松的,即其中数据分布是非平稳的并且随着时间而变化。本文用上下文变量$c$表示数据分布状态。$c$中的漂移导致数据分布漂移。上下文漂移可以改变目标分布、输入分布或两者。此外,分布漂移可能是突然的或逐渐的。在持续学习中,上下文漂移可能会干扰学习过程并擦除先前学习的知识;因此,持续学习算法必须包括专门的机制来处理这种漂移。在本文中,我们的目标是识别和分类不同类型的上下文漂移和潜在的假设,以更好地描述各种持续学习的情况。此外,我们建议使用分布漂移框架,以提供更精确的定义,在持续学习领域中常用的几个术语。

英文摘要:

Classical machine learning algorithms often assume that the data are drawn i.i.d. from a stationary probability distribution. Recently, continual learning emerged as a rapidly growing area of machine learning where this assumption is relaxed, i.e. where the data distribution is non-stationary and changes over time. This paper represents the state of data distribution by a context variable $c$. A drift in $c$ leads to a data distribution drift. A context drift may change the target distribution, the input distribution, or both. Moreover, distribution drifts might be abrupt or gradual. In continual learning, context drifts may interfere with the learning process and erase previously learned knowledge; thus, continual learning algorithms must include specialized mechanisms to deal with such drifts. In this paper, we aim to identify and categorize different types of context drifts and potential assumptions about them, to better characterize various continual-learning scenarios. Moreover, we propose to use the distribution drift framework to provide more precise definitions of several terms commonly used in the continual learning field.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑