自蒸馏使持续学习成为可能
Self-Distillation Enables Continual Learning
AI总结：
自蒸馏微调通过直接从演示中进行基于策略的学习，有效减少持续学习中的灾难性遗忘，提升新任务准确性。
摘要：
持续学习，使模型能够在不损害现有能力的情况下获取新技能和知识，仍然是基础模型面临的基本挑战。虽然基于策略的强化学习可以减少遗忘，但需要显式的奖励函数，这些函数通常不可用。从专家演示中学习，是主要的替代方法，但被监督微调（SFT）所主导，而SFT本质上是离策略的。我们引入了自蒸馏微调（SDFT），一种简单的方法，使模型能够直接从演示中进行基于策略的学习。SDFT通过上下文学习利用演示条件化的模型作为自己的教师，生成基于策略的训练信号，在保留先前能力的同时获取新技能。在技能学习和知识获取任务中，SDFT始终优于SFT，实现了更高的新任务准确性，同时显著减少了灾难性遗忘。在连续学习实验中，SDFT使单个模型在时间推移过程中积累多种技能而不出现性能退化，确立了基于策略蒸馏作为从演示中进行持续学习的实用路径。
英文摘要：
Continual learning, enabling models to acquire new skills and knowledge without degrading existing capabilities, remains a fundamental challenge for foundation models. While on-policy reinforcement learning can reduce forgetting, it requires explicit reward functions that are often unavailable. Learning from expert demonstrations, the primary alternative, is dominated by supervised fine-tuning (SFT), which is inherently off-policy. We introduce Self-Distillation Fine-Tuning (SDFT), a simple method that enables on-policy learning directly from demonstrations. SDFT leverages in-context learning by using a demonstration-conditioned model as its own teacher, generating on-policy training signals that preserve prior capabilities while acquiring new skills. Across skill learning and knowledge acquisition tasks, SDFT consistently outperforms SFT, achieving higher new-task accuracy while substantially reducing catastrophic forgetting. In sequential learning experiments, SDFT enables a single model to accumulate multiple skills over time without performance regression, establishing on-policy distillation as a practical path to continual learning from demonstrations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
