超越经验回放与完整模型代理的持续学习
Continual Learning Beyond Experience Rehearsal and Full Model Surrogates
AI总结：
提出SPARC持续学习方法，通过结合任务特定工作记忆与任务无关语义记忆，无需经验回放或完整模型代理，仅用6%参数实现优越性能并缓解任务偏差。
摘要：
持续学习（CL）一直是深度神经网络面临的重大挑战，因为学习新任务会部分或完全抹去先前获得的知识。现有解决方案通常依赖经验回放或完整模型代理来缓解灾难性遗忘。虽然有效，但这些方法引入了大量内存和计算开销，限制了其在现实场景中的可扩展性和适用性。为此，我们提出SPARC，一种可扩展的CL方法，消除了对经验回放和完整模型代理的需求。通过有效结合任务特定工作记忆和任务无关语义记忆进行跨任务知识整合，SPARC实现了显著的参数效率，仅使用完整模型代理所需参数的6%。尽管设计轻量，SPARC在Seq-TinyImageNet上取得优越性能，并在多个CL基准上匹敌基于回放的方法。此外，分类层中的权重重归一化缓解了任务特定偏差，使SPARC成为在严格效率约束下CL的实用且可扩展的解决方案。
英文摘要：
Continual learning (CL) has remained a significant challenge for deep neural networks as learning new tasks erases previously acquired knowledge, either partially or completely. Existing solutions often rely on experience rehearsal or full model surrogates to mitigate CF. While effective, these approaches introduce substantial memory and computational overhead, limiting their scalability and applicability in real-world scenarios. To address this, we propose SPARC, a scalable CL approach that eliminates the need for experience rehearsal and full-model surrogates. By effectively combining task-specific working memories and task-agnostic semantic memory for cross-task knowledge consolidation, SPARC results in a remarkable parameter efficiency, using only 6% of the parameters required by full-model surrogates. Despite its lightweight design, SPARC achieves superior performance on Seq-TinyImageNet and matches rehearsal-based methods on various CL benchmarks. Additionally, weight re-normalization in the classification layer mitigates task-specific biases, establishing SPARC as a practical and scalable solution for CL under stringent efficiency constraints.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
