SuRe: 基于惊喜的优先回放用于连续LLM学习
SuRe: Surprise-Driven Prioritised Replay for Continual LLM Learning
AI总结：
SuRe通过基于惊喜的优先回放和双学习者设计，提升连续LLM学习的性能和稳定性，有效缓解灾难性遗忘问题。
摘要：
连续学习，即在一系列任务中适应而不断忘却之前获得的知识的能力，仍然是机器学习中的主要挑战，也是人工智能与人类智能之间的关键差距。尽管正则化和回放方法在视觉领域表现良好，但在大规模语言模型（LLMs）中，它们在多任务学习中的表现却滞后，尤其是在任务数量众多的情况下。我们重新审视回放方法，并认为两种失败模式导致了这一差距：选择（要回放什么）和整合（如何巩固新知识）。为了解决选择问题，我们提出了惊喜优先回放（SuRe），一种简单、架构无关的规则，用于对最令人惊讶（高负对数似然）的序列进行排序和存储。SuRe在大量任务（LNT）设置中实现了最先进的性能，并在标准CL和LNT基准测试中提供了最佳的总体平均值。为了解决整合问题，我们添加了双学习者设计，通过指数移动平均（EMA）合并快速和缓慢的LoRA适配器，使快速适应同时稳定长期知识。将SuRe与双学习者结合，进一步提高了性能，包括在LNT上比先前SOTA提高了多达+5个准确率点。消融研究证实，我们提出的方法在减少回放频率和小缓冲区大小的情况下仍然稳健，证明了其有效性和样本效率。总的来说，我们的结果将回放确立为连续LLM微调的强大基线，并展示了基于惊喜的选择和缓慢权重整合是缓解灾难性遗忘的互补组件。
英文摘要：
Continual learning, one's ability to adapt to a sequence of tasks without forgetting previously acquired knowledge, remains a major challenge in machine learning and a key gap between artificial and human intelligence. While regularisation and replay perform well in vision, they lag behind multi-task learning for large language models (LLMs), especially at scale with many tasks. We revisit replay and argue that two failure modes drive this gap: selection (what to rehearse) and integration (how to consolidate new knowledge). To address selection, we propose Surprise-prioritised Replay (SuRe), a simple, architecture-agnostic rule that ranks and stores the most surprising (high Negative Log-Likelihood) sequences. SuRe achieves state-of-the-art performance in the Large Number of Tasks (LNT) setting and delivers the best overall average across both Standard CL and LNT benchmarks. To address integration, we add a dual-learner design with fast and slow LoRA adapters merged via an exponential moving average (EMA), enabling rapid adaptation while stabilising long-term knowledge. Combining SuRe with the dual learner yields further gains, including improvements of up to +5 accuracy points on LNT over prior SOTA. Ablation studies confirm that our proposed method remains robust under reduced replay frequency and small buffer size, demonstrating both effectiveness and sample efficiency. Taken together, our results establish replay as a strong baseline for continual LLM fine-tuning and demonstrate that surprise-based selection and slow-weight consolidation are complementary components for mitigating catastrophic forgetting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
