遗忘遗忘：在充足记忆世界中的持续学习
Forget Forgetting: Continual Learning in a World of Abundant Memory
AI总结：
本文提出了一种轻量级方法，通过权重空间巩固在充足内存环境下提升持续学习性能，挑战传统假设并提供低计算成本的替代方案。
摘要：
持续学习（CL）传统上专注于最小化示例记忆，这是一个常常与现代系统不一致的约束，其中GPU时间而不是存储是主要瓶颈。本文挑战这一范式，通过研究一个更现实的场景：一个内存充足到足以缓解遗忘，但完全从头重新训练仍然成本过高的场景。在这个实用的“中间地带”，我们发现核心挑战从稳定性转向了可塑性，因为模型倾向于先前任务而难以学习新任务。相反，改进的稳定性使简单的重放基线在GPU成本仅为现有方法分数的情况下优于最先进方法。为了解决这一新出现的权衡，我们提出了权重空间巩固，一种轻量级方法，结合（1）基于秩的参数重置以恢复可塑性与（2）权重平均以增强稳定性。在类别递增学习和持续指令微调中验证，我们的方法在性能上优于强基线，同时匹配重放的低成本，提供了一种可扩展的替代方案，以替代昂贵的全重新训练。这些发现挑战了长期持续学习的假设，并为现实世界持续学习系统建立了新的、成本效益高的基线，其中示例记忆不再是限制因素。
英文摘要：
Continual learning (CL) has traditionally focused on minimizing exemplar memory, a constraint often misaligned with modern systems where GPU time, not storage, is the primary bottleneck. This paper challenges this paradigm by investigating a more realistic regime: one where memory is abundant enough to mitigate forgetting, but full retraining from scratch remains prohibitively expensive. In this practical "middle ground", we find that the core challenge shifts from stability to plasticity, as models become biased toward prior tasks and struggle to learn new ones. Conversely, improved stability allows simple replay baselines to outperform the state-of-the-art methods at a fraction of the GPU cost. To address this newly surfaced trade-off, we propose Weight Space Consolidation, a lightweight method that combines (1) rank-based parameter resets to restore plasticity with (2) weight averaging to enhance stability. Validated on both class-incremental learning with image classifiers and continual instruction tuning with large language models, our approach outperforms strong baselines while matching the low computational cost of replay, offering a scalable alternative to expensive full-retraining. These findings challenge long-standing CL assumptions and establish a new, cost-efficient baseline for real-world CL systems where exemplar memory is no longer the limiting factor.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
