持续学习中基于记忆的方法的泛化研究
Studying Generalization on Memory-Based Methods in Continual Learning
AI总结：
本文研究持续学习中基于记忆的方法的泛化问题，发现它们虽有利于分布内泛化，但会因学习虚假特征而损害分布外泛化，且问题主要出现在线性分类器上。
摘要：
持续学习的目标之一是在不断到来的经验流中持续学习新概念，同时避免灾难性遗忘。为了减轻完全的知识覆盖，基于记忆的方法存储一定比例的先前数据分布，用于训练过程中。尽管这些方法产生了良好的结果，但很少有研究测试它们的分布外泛化特性，以及这些方法是否过拟合了回放记忆。在这项工作中，我们表明，尽管这些方法有助于传统的分布内泛化，但它们可能通过学习虚假特征和相关性而严重损害分布外泛化。利用受控环境，即Synbol基准生成器（Lacoste等人，2020），我们证明了这种分布外泛化的缺乏主要发生在线性分类器中。
英文摘要：
One of the objectives of Continual Learning is to learn new concepts continually over a stream of experiences and at the same time avoid catastrophic forgetting. To mitigate complete knowledge overwriting, memory-based methods store a percentage of previous data distributions to be used during training. Although these methods produce good results, few studies have tested their out-of-distribution generalization properties, as well as whether these methods overfit the replay memory. In this work, we show that although these methods can help in traditional in-distribution generalization, they can strongly impair out-of-distribution generalization by learning spurious features and correlations. Using a controlled environment, the Synbol benchmark generator (Lacoste et al., 2020), we demonstrate that this lack of out-of-distribution generalization mainly occurs in the linear classifier.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
