通过上下文蒸馏更新参数化知识可保留训练后能力
Updating Parametric Knowledge with Context Distillation Retains Post-Training Capabilities
AI总结：
本文提出DiSC方法，通过上下文蒸馏实现持续知识适应，有效平衡新知识学习与原有能力保持。
摘要：
训练后的预训练大语言模型具备多种有益技能，包括遵循指令、推理等。然而，这些训练后的LLM只编码到截止日期的知识，需要持续适应。不幸的是，现有解决方案无法同时从适应文档语料中学习新知识并减轻先前学习能力的遗忘。为此，我们引入了通过分割上下文的蒸馏（DiSC），一种基于上下文蒸馏的持续知识适应方法。DiSC通过在训练示例的不同段上条件化学生和教师分布，并最小化共享标记之间的KL散度，从而高效地应用上下文蒸馏，而无需在训练过程中进行显式生成步骤。我们在四个训练后的模型和两个适应领域上进行了实验。与先前的微调和蒸馏方法相比，DiSC始终在学习新知识和减轻先前学习技能（如指令遵循、推理和事实知识）的遗忘之间取得了最佳平衡。
英文摘要：
Post-training endows pretrained LLMs with a variety of desirable skills, including instruction-following, reasoning, and others. However, these post-trained LLMs only encode knowledge up to a cut-off date, necessitating continual adaptation. Unfortunately, existing solutions cannot simultaneously learn new knowledge from an adaptation document corpora and mitigate the forgetting of earlier learned capabilities. To address this, we introduce Distillation via Split Contexts (DiSC), a simple context-distillation based approach for continual knowledge adaptation. \methodname~derives student and teacher distributions by conditioning on distinct segments of the training example and minimizes the KL divergence between the shared tokens. This allows us to efficiently apply context-distillation without requiring explicit generation steps during training. We run experiments on four post-trained models and two adaptation domains. Compared to prior finetuning and distillation methods for continual adaptation, DiSC consistently reports the best trade-off between learning new knowledge and mitigating forgetting of previously learned skills like instruction-following, reasoning, and factual knowledge.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
