持续学习中虚假遗忘的实时检测与定量分析
Real Time Detection and Quantitative Analysis of Spurious Forgetting in Continual Learning
AI总结：
本文提出浅层与深层对齐框架，通过定量指标和实时检测方法解决持续学习中的虚假遗忘问题，提升模型鲁棒性。
摘要：
灾难性遗忘仍然是大语言模型持续学习中的基本挑战。近期研究发现，性能下降可能源于任务对齐破坏导致的虚假遗忘，而非真正的知识损失。然而，此前工作仅定性描述对齐情况，依赖事后分析，缺乏自动区分机制。我们引入浅层与深层对齐框架，提供首个对齐深度的定量表征。我们发现当前任务对齐方法存在浅层对齐问题——仅维持在前几个输出token（约3-5个）上，使模型易受遗忘影响。这解释了虚假遗忘为何发生、为何可逆以及为何微调攻击有效。我们提出一个综合框架解决所有缺口：（1）定量指标（0-1尺度）用于测量不同token位置的对齐深度；（2）实时检测方法用于训练过程中识别浅层对齐；（3）专用分析工具用于可视化和恢复预测；（4）适应性缓解策略可自动区分遗忘类型并促进深层对齐。在多个数据集和模型架构（Qwen2.5-3B到Qwen2.5-32B）上的广泛实验显示，识别准确率达到86.2-90.6%，表明促进深层对齐可使模型对遗忘的鲁棒性提升3.3-7.1%。
英文摘要：
Catastrophic forgetting remains a fundamental challenge in continual learning for large language models. Recent work revealed that performance degradation may stem from spurious forgetting caused by task alignment disruption rather than true knowledge loss. However, this work only qualitatively describes alignment, relies on post-hoc analysis, and lacks automatic distinction mechanisms. We introduce the shallow versus deep alignment framework, providing the first quantitative characterization of alignment depth. We identify that current task alignment approaches suffer from shallow alignment - maintained only over the first few output tokens (approximately 3-5) - making models vulnerable to forgetting. This explains why spurious forgetting occurs, why it is reversible, and why fine-tuning attacks are effective. We propose a comprehensive framework addressing all gaps: (1) quantitative metrics (0-1 scale) to measure alignment depth across token positions; (2) real-time detection methods for identifying shallow alignment during training; (3) specialized analysis tools for visualization and recovery prediction; and (4) adaptive mitigation strategies that automatically distinguish forgetting types and promote deep alignment. Extensive experiments on multiple datasets and model architectures (Qwen2.5-3B to Qwen2.5-32B) demonstrate 86.2-90.6% identification accuracy and show that promoting deep alignment improves robustness against forgetting by 3.3-7.1% over baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
