只取所需：秩最小化作为持续学习中的隐式遗忘正则化器
Take Only What You Need: Rank Minimization as an Implicit Forgetting Regularizer in Continual Learning
AI总结：
本文提出CoDyRA方法，通过秩最小化作为隐式遗忘正则化器，在持续学习中平衡可塑性与稳定性，在多个基准上优于现有方法。
摘要：
持续学习中的核心张力是可塑性（获取新知识）与稳定性（保留先前知识）之间的权衡。我们研究如何通过容量控制（即调节每次参数更新的有效秩，这是LoRA更新中可直接控制的逐步骤量）来持续更新预训练骨干网络，使其吸收新知识的同时保留现有能力。对模块和任务间LoRA秩和放置的受控探测揭示了一致的权衡，存在一个随放置和任务变化的中等秩最佳点，没有普遍最优的固定秩；一个形式化界限表明遗忘随秩增长。基于这些发现，我们提出了持续动态秩选择LoRA（CoDyRA），该方法通过在每个组件重要性权重上施加稀疏性促进正则化，联合训练每个LoRA更新与秩最小化。监督目标驱动可塑性；秩最小化正则化遗忘。我们证明秩最小化在持续学习机制中充当隐式遗忘正则化器，通过控制相对于当前模型状态的遗忘，同时保护通用能力和先前任务知识。在MTIL、X-TAIL和TRACE（CLIP、LLaMA、Gemma）上，CoDyRA在新知识学习和遗忘方面优于先前的持续学习方法，实现了强大的可塑性-稳定性平衡。代码可在https://github.com/jeff024/codyra获取。
英文摘要：
The central tension in continual learning (CL) is the trade-off between plasticity (acquiring new knowledge) and stability (retaining prior knowledge). We study how a pre-trained backbone can be continually updated to absorb new knowledge while preserving existing capabilities, via capacity control: regulating the effective rank of each parameter update, a per-step quantity directly controllable inside a LoRA update. A controlled probe of LoRA rank and placement across modules and tasks reveals a consistent trade-off, with a moderate-rank sweet spot that varies by placement and task, leaving no universally optimal fixed rank; a formal bound shows forgetting grows with rank. Building on these findings, we propose Continual Dynamic Rank-Selective LoRA (CoDyRA), which jointly trains each LoRA update with rank minimization via sparsity-promoting regularization on per-component importance weights. The supervised objective drives plasticity; rank minimization regularizes forgetting. We show that rank minimization serves as an implicit forgetting regularizer in the CL regime, protecting general capability and prior-task knowledge simultaneously by controlling forgetting against the current model state. Across MTIL, X-TAIL, and TRACE (CLIP, LLaMA, Gemma), CoDyRA outperforms prior CL methods on new knowledge learning and forgetting, achieving a strong plasticity-stability balance. Project page: https://artificer-ai-lab.github.io/CoDyRA.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
