在持续学习中通过闪回协调稳定性和可塑性
Flashbacks to Harmonize Stability and Plasticity in Continual Learning
AI总结：
本文提出Flashback Learning (FL)方法，通过双向正则化和双知识库设计协调持续学习中的稳定性与可塑性，在标准基准上显著提升准确率并超越现有方法。
摘要：
我们引入了Flashback Learning (FL)，这是一种旨在协调持续学习 (CL) 中模型稳定性和可塑性的新方法。与以往主要侧重于正则化模型更新以在学习新概念时保留旧信息的方法不同，FL通过双向正则化显式地平衡这一权衡。这种方法有效地引导模型在积极保留旧知识的同时迅速吸收新知识。FL通过两阶段训练过程进行操作，可以无缝集成到各种CL方法中，包括重放、参数正则化、蒸馏和动态架构技术。在设计FL时，我们使用了两个不同的知识库：一个用于增强可塑性，另一个用于提高稳定性。FL通过利用这两个知识库来正则化模型更新，确保模型更加平衡。在理论上，我们分析了FL机制如何增强稳定性-可塑性平衡。在经验上，FL在相同的训练预算内相对于基线方法表现出切实的改进。通过将FL集成到每个CL类别中至少一个代表性基线中，我们在标准图像分类基准上观察到，在类增量设置中平均准确率提高达4.91%，在任务增量设置中提高达3.51%。此外，稳定性与可塑性比率的测量证实FL有效增强了这种平衡。FL还在更具挑战性的数据集（如ImageNet）上优于最先进的CL方法。
英文摘要：
We introduce Flashback Learning (FL), a novel method designed to harmonize the stability and plasticity of models in Continual Learning (CL). Unlike prior approaches that primarily focus on regularizing model updates to preserve old information while learning new concepts, FL explicitly balances this trade-off through a bidirectional form of regularization. This approach effectively guides the model to swiftly incorporate new knowledge while actively retaining its old knowledge. FL operates through a two-phase training process and can be seamlessly integrated into various CL methods, including replay, parameter regularization, distillation, and dynamic architecture techniques. In designing FL, we use two distinct knowledge bases: one to enhance plasticity and another to improve stability. FL ensures a more balanced model by utilizing both knowledge bases to regularize model updates. Theoretically, we analyze how the FL mechanism enhances the stability-plasticity balance. Empirically, FL demonstrates tangible improvements over baseline methods within the same training budget. By integrating FL into at least one representative baseline from each CL category, we observed an average accuracy improvement of up to 4.91% in Class-Incremental and 3.51% in Task-Incremental settings on standard image classification benchmarks. Additionally, measurements of the stability-to-plasticity ratio confirm that FL effectively enhances this balance. FL also outperforms state-of-the-art CL methods on more challenging datasets like ImageNet.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
