当前的持续学习方法真的不可知吗？引入OPRE，迈向不可知持续学习的一步
Are Current Continual Learning Methods Truly Agnostic? Introducing OPRE, a Step Toward Agnostic Continual Learning
AI总结：
研究持续学习中灾难性遗忘问题，指出多数方法有先验信息非不可知。提出在线补丁冗余消除器OPRE，通过输入空间标准丢弃信息，在CIFAR 10和CIFAR - 100上性能良好，优于GDumb，为持续学习提供新视角。
摘要：
为实现持续学习（CL），灾难性遗忘问题必须被克服，这一问题自神经网络诞生以来就一直困扰着它。持续学习方法的评估依赖于分割已知的同质数据集并依次学习相关任务。我们认为大多数CL方法引入了关于未来数据的先验信息，不能被视为不可知。以依赖预训练特征提取器的方法为例进行说明，这种方法在CL中仍被使用。在表明预训练特征提取器意味着模型对可学习数据的通用性丧失后，我们讨论了其他CL方法中引入的其他先验信息。然后我们提出了在线补丁冗余消除器（OPRE），一种通过两个明确的输入空间标准丢弃信息的在线数据集压缩算法。在测试时使用随机初始化的分类器，OPRE在CIFAR 10和CIFAR - 100上的性能与已报道的在线持续学习方法相匹配，且无需任何预训练特征提取器，在相同内存预算下优于GDumb，同时对未来数据仅做了最小且可解释的假设。我们将这些结果构建为关于持续学习的经验性、信息论视角。
英文摘要：
In order to achieve Continual Learning (CL), the problem of catastrophic forgetting, one that has plagued neural networks since their inception, must be overcome. The evaluation of continual learning methods relies on splitting a known homogeneous dataset and learning the associated tasks one after the other. We argue that most CL methods introduce a priori information about the data to come and cannot be considered agnostic. We exemplify this point with the case of methods relying on pretrained feature extractors, which are still used in CL. After showing that pretrained feature extractors imply a loss of generality with respect to the data that can be learned by the model, we then discuss other kinds of a priori information introduced in other CL methods. We then present the Online Patch Redundancy Eliminator (OPRE), an online dataset-compression algorithm that discards information through two explicit, input-space criteria. With a classifier that was randomly initialized at test time, OPRE's performance matches reported state-of-the-art online continual-learning methods on CIFAR 10 and CIFAR-100 without any pretrained feature extractor, and outperforms GDumb at an identical memory budget-while making only minimal and interpretable assumptions about the data to come. We frame these results as an empirical, information-theoretic perspective on continual learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
