自适应保留与纠正：持续学习的测试时训练
Adaptive Retention & Correction: Test-Time Training for Continual Learning
AI总结：
针对持续学习中分类层偏向最近任务的问题，提出基于测试时训练的ARC方法，通过任务外检测、自适应保留与纠正机制，无需修改训练过程即可集成到现有方法，在CIFAR-100和Imagenet-R上分别提升2.7%和2.6%。
摘要：
持续学习，也称为终身学习或增量学习，是指模型随时间从不断到来的数据流中学习的过程。持续学习中的一个常见问题是分类层对最近任务的偏置。传统上，方法依赖于在训练期间纳入过去任务的数据来缓解这一问题。然而，近期持续学习向无记忆环境的转变使得这些方法变得不可行。在本研究中，我们提出了一种专注于测试阶段的解决方案。我们首先引入了一种简单的任务外检测方法OTD，旨在测试期间准确识别来自过去任务的样本。利用OTD，我们随后提出：（1）一种自适应保留机制，用于在过去的任务数据上动态调整分类器层；（2）一种自适应纠正机制，用于当模型将来自先前任务的数据分类到当前任务的类别时修正预测。我们将我们的方法命名为自适应保留与纠正（ARC）。虽然ARC是为无记忆环境设计的，但它在基于记忆的设置中也证明有效。大量实验表明，我们的方法可以插入到几乎任何现有的持续学习方法中，而无需对其训练过程进行任何修改。具体来说，当与最先进的方法集成时，ARC在CIFAR-100和Imagenet-R数据集上分别实现了平均性能提升2.7%和2.6%。
英文摘要：
Continual learning, also known as lifelong learning or incremental learning, refers to the process by which a model learns from a stream of incoming data over time. A common problem in continual learning is the classification layer's bias towards the most recent task. Traditionally, methods have relied on incorporating data from past tasks during training to mitigate this issue. However, the recent shift in continual learning to memory-free environments has rendered these approaches infeasible. In this study, we propose a solution focused on the testing phase. We first introduce a simple Out-of-Task Detection method, OTD, designed to accurately identify samples from past tasks during testing. Leveraging OTD, we then propose: (1) an Adaptive Retention mechanism for dynamically tuning the classifier layer on past task data; (2) an Adaptive Correction mechanism for revising predictions when the model classifies data from previous tasks into classes from the current task. We name our approach Adaptive Retention & Correction (ARC). While designed for memory-free environments, ARC also proves effective in memory-based settings. Extensive experiments show that our proposed method can be plugged in to virtually any existing continual learning approach without requiring any modifications to its training procedure. Specifically, when integrated with state-of-the-art approaches, ARC achieves an average performance increase of 2.7% and 2.6% on the CIFAR-100 and Imagenet-R datasets, respectively.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
