通过条件锚定蒸馏在持续学习中稳定语言模型

中文摘要 AI 辅助

语言模型的持续适应可能会改变其在先前学习提示上的输出分布，而保留每一个旧的提示-答案对可能是不期望的或不可能的。我们研究了条件锚定生成蒸馏（CAGD）：保留一小部分旧提示，使用冻结的先前模型重建补全和生成状态，并在学习下一个任务时匹配其预测分布。该公式区分了普通回放混淆的三个角色：条件选择要保护的行为，教师生成定位相关状态，软目标指定预测可能如何变化。对于自回归语言生成，教师展开蒸馏允许序列散度的精确链式法则分解。对于掩码扩散语言建模，我们的实现直接控制教师生成补全上的局部去噪漂移。在219M掩码扩散语言模型的持续适应中，CAGD在一个任务顺序上将四项最终保留损失从2.927降至1.114，在完全相反的顺序中从2.168降至0.891。当教师生成的支持保持相同时，相同的软目标在硬回放上将最终平均损失降低0.055。该方向在SMDM和Qwen3上的新事实和自然指令中持续存在。在GSM8K上，Qwen适应保持了答案格式合规性，但精确匹配保留在0.6B时种子混合，在1.7B时恶化。这些结果支持条件锚定功能保留作为跨测试的语言生成目标的共同设计原则。

英文摘要

Continual adaptation of language models can change their output distribution on prompts learned earlier, while retaining every old prompt-answer pair may be undesirable or impossible. We study condition-anchored generative distillation (CAGD): retain a small set of old prompts, use a frozen previous model to reconstruct completions and generation states, and match its predictive distributions while learning the next task. The formulation separates three roles that ordinary replay conflates: conditions select the behavior to protect, teacher generations locate relevant states, and soft targets specify how predictions may change. For autoregressive language generation, teacher-rollout distillation admits an exact chain-rule decomposition of sequence divergence. For masked-diffusion language modeling, our implementation directly controls local denoising drift on teacher-generated completions. In continual adaptation of a 219M masked diffusion language model, CAGD reduces four-task final held-out loss from 2.927 to 1.114 in one task order and from 2.168 to 0.891 in exact reverse. The same soft targets lower final average loss by 0.055 over hard replay when teacher-generated support is held identical. The direction persists on fresh facts and natural instructions across SMDM and Qwen3. On GSM8K, Qwen adaptation preserves answer-format compliance, but exact-match retention is seed-mixed at 0.6B and worsens at 1.7B. These results support condition-anchored functional preservation as a common design principle across the tested language-generation objectives.

发表机构

• Institute for AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院（AIR））
• Nanjing University（南京大学）

机构由 AI 辅助整理，请以论文原文为准。