解开语言模型遗忘之谜：低秩示例关联
Demystifying Language Model Forgetting with Low-rank Example Associations
AI总结：
本文通过低秩矩阵分析揭示语言模型微调时遗忘的机制，并提出基于矩阵补全的方法有效缓解遗忘问题。
摘要：
大型语言模型（LLMs）在微调时会遗忘上游知识。尽管已有努力缓解遗忘问题，但很少有人研究遗忘的上游示例如何依赖于新学习的任务。对这些依赖关系的洞察能够实现高效且有针对性的遗忘缓解。在本文中，我们实证分析了在微调LLMs在一个新的任务之一（M个新任务中的一个）后，N个上游语言建模或指令微调示例中发生的遗忘现象，并以M×N矩阵可视化。我们显示这些矩阵往往可以很好地用低秩矩阵近似，表明学习任务与遗忘的上游示例之间存在简单的关联。利用这一分析，我们通过矩阵补全来预测在微调LLMs到未见过的任务时的遗忘情况，这使我们能够在不昂贵地对整个上游数据进行推理的情况下快速识别出最被遗忘的示例。尽管方法简单，但其在学习学习任务和上游示例之间的语义关系方面优于先前的方法。我们通过展示在微调过程中对预测示例进行重新学习时统计上显著减少的遗忘来证明我们分析的实用性。代码、数据和统计数据收集：https://github.com/AuCson/low-rank-forgetting
英文摘要：
Large language models (LLMs) suffer from forgetting of upstream knowledge when fine-tuned. Despite efforts on mitigating forgetting, few have investigated how forgotten upstream examples are dependent on newly learned tasks. Insights on such dependencies enable efficient and targeted mitigation of forgetting. In this paper, we empirically analyze forgetting that occurs in $N$ upstream examples of language modeling or instruction-tuning after fine-tuning LLMs on one of $M$ new tasks, visualized in $M\times N$ matrices. We show that the matrices are often well-approximated with low-rank matrices, indicating the dominance of simple associations between the learned tasks and forgotten upstream examples. Leveraging the analysis, we predict forgetting of upstream examples when fine-tuning LLMs on unseen tasks with matrix completion over the empirical associations. This enables fast identification of most forgotten examples without expensive inference on the entire upstream data. Despite simplicity, the approach outperforms prior approaches that learn semantic relationships of learned tasks and upstream examples with LMs. We demonstrate the practical utility of our analysis by showing statistically significantly reduced forgetting as we upweight predicted examples for replay during fine-tuning. Code, data, and statistics collected: https://github.com/AuCson/low-rank-forgetting
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
