MSSR: 为持续LLM微调的内存感知自适应回放
MSSR: Memory-Aware Adaptive Replay for Continual LLM Fine-Tuning
AI总结：
MSSR通过估计样本级别的记忆强度并自适应地安排复习，有效缓解持续LLM微调中的灾难性遗忘问题，同时保持快速适应能力。
摘要：
大型语言模型（LLM）的持续微调正变得越来越关键，因为这些模型被部署在动态环境中，任务和数据分布随时间推移而演变。尽管强大的适应性使模型能够快速获取新知识，但也使LLM暴露于灾难性遗忘的风险中，即在连续训练过程中先前学习的技能会退化。现有的回放策略，如固定交错回放、准确性监督和损失驱动调度，仍存在局限：一些依赖启发式规则，只能部分缓解遗忘，而其他方法虽然提高了性能，但带来了显著的计算开销。受连续微调过程中保留动态的启发，我们提出了一种名为记忆启发采样器和调度器回放（MSSR）的体验回放框架，该框架估计样本级别的记忆强度，并在自适应间隔内安排复习，以在保持快速适应的同时缓解灾难性遗忘。在三个主干模型和11个连续任务上的广泛实验表明，MSSR在持续微调中一致优于最先进的回放基线，尤其在需要推理和多选基准测试中表现突出。
英文摘要：
Continual fine-tuning of large language models (LLMs) is becoming increasingly crucial as these models are deployed in dynamic environments where tasks and data distributions evolve over time. While strong adaptability enables rapid acquisition of new knowledge, it also exposes LLMs to catastrophic forgetting, where previously learned skills degrade during sequential training. Existing replay-based strategies, such as fixed interleaved replay, accuracy-supervised, and loss-driven scheduling, remain limited: some depend on heuristic rules and provide only partial mitigation of forgetting, while others improve performance but incur substantial computational overhead. Motivated by retention dynamics under sequential fine-tuning, we propose Memory-Inspired Sampler and Scheduler Replay (MSSR), an experience replay framework that estimates sample-level memory strength and schedules rehearsal at adaptive intervals to mitigate catastrophic forgetting while maintaining fast adaptation. Extensive experiments across three backbone models and 11 sequential tasks show that MSSR consistently outperforms state-of-the-art replay baselines, with particularly strong gains on reasoning-intensive and multiple-choice benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
