联合回溯适应用于抗遗忘的指令微调
Joint Flashback Adaptation for Forgetting-Resistant Instruction Tuning
AI总结：
本文提出联合回溯适应方法，通过引入旧任务的有限提示来缓解增量学习中的灾难性遗忘问题，提升模型在新任务上的泛化能力。
摘要：
大型语言模型在各种任务中取得了显著成功。然而，由于灾难性遗忘，它们在学习新任务时面临挑战。现有方法依赖于经验回放、优化约束或任务区分，但在现实场景中受限颇多。为了解决这些问题，我们提出了联合回溯适应。我们首先在适应新任务时引入回溯——即从旧任务中选取有限数量的提示，并限制模型输出与原始输出的偏差。然后在回溯和新任务之间插值潜在任务，以实现同时学习相关潜在任务、新任务和回溯，缓解回溯中的数据稀疏性，并促进知识共享以实现平滑适应。我们的方法仅需少量回溯，无需访问回放数据，并且是任务无关的。我们在1000多个指令遵循任务、算术推理任务和一般推理任务上对最先进的大型语言模型进行了广泛实验。结果表明，我们的方法在提高新任务上的泛化能力和减少旧任务上的遗忘方面表现优异。
英文摘要：
Large language models have achieved remarkable success in various tasks. However, it is challenging for them to learn new tasks incrementally due to catastrophic forgetting. Existing approaches rely on experience replay, optimization constraints, or task differentiation, which encounter strict limitations in real-world scenarios. To address these issues, we propose Joint Flashback Adaptation. We first introduce flashbacks -- a limited number of prompts from old tasks -- when adapting to new tasks and constrain the deviations of the model outputs compared to the original one. We then interpolate latent tasks between flashbacks and new tasks to enable jointly learning relevant latent tasks, new tasks, and flashbacks, alleviating data sparsity in flashbacks and facilitating knowledge sharing for smooth adaptation. Our method requires only a limited number of flashbacks without access to the replay data and is task-agnostic. We conduct extensive experiments on state-of-the-art large language models across 1000+ instruction-following tasks, arithmetic reasoning tasks, and general reasoning tasks. The results demonstrate the superior performance of our method in improving generalization on new tasks and reducing forgetting in old tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
