SEEKR：用于大型语言模型持续学习的选择性注意力引导知识保留
SEEKR: Selective Attention-Guided Knowledge Retention for Continual Learning of Large Language Models
AI总结：
针对持续学习中的灾难性遗忘问题，提出SEEKR方法，通过选择性注意力蒸馏保留先前知识，仅需1/10重放数据即实现更优性能。
摘要：
持续学习（CL）对于语言模型动态适应不断演变的现实需求至关重要。为了缓解CL中的灾难性遗忘问题，数据重放已被证明是一种简单且有效的策略，随后基于数据重放的蒸馏可以进一步提升性能。然而，现有方法未能充分利用模型从先前任务中嵌入的知识，导致需要相对大量的重放样本才能取得良好效果。在本研究中，我们首先探索并强调注意力权重在知识保留中的重要性，然后提出了一种选择性注意力引导知识保留方法（SEEKR），用于大型语言模型（LLMs）的数据高效重放持续学习。具体而言，SEEKR对选定的注意力头执行注意力蒸馏以实现更细粒度的知识保留，其中提出的基于可遗忘性和基于任务敏感性的度量用于识别最有价值的注意力头。在两个LLM持续学习基准上的实验结果证明了SEEKR在性能和效率上均优于现有方法。具体而言，SEEKR仅使用其他方法1/10的重放数据就实现了相当甚至更好的性能，并将重放数据的比例降低至1%。
英文摘要：
Continual learning (CL) is crucial for language models to dynamically adapt to the evolving real-world demands. To mitigate the catastrophic forgetting problem in CL, data replay has been proven a simple and effective strategy, and the subsequent data-replay-based distillation can further enhance the performance. However, existing methods fail to fully exploit the knowledge embedded in models from previous tasks, resulting in the need for a relatively large number of replay samples to achieve good results. In this work, we first explore and emphasize the importance of attention weights in knowledge retention, and then propose a SElective attEntion-guided Knowledge Retention method (SEEKR) for data-efficient replay-based continual learning of large language models (LLMs). Specifically, SEEKR performs attention distillation on the selected attention heads for finer-grained knowledge retention, where the proposed forgettability-based and task-sensitivity-based measures are used to identify the most valuable attention heads. Experimental results on two continual learning benchmarks for LLMs demonstrate the superiority of SEEKR over the existing methods on both performance and efficiency. Explicitly, SEEKR achieves comparable or even better performance with only 1/10 of the replayed data used by other methods, and reduces the proportion of replayed data to 1%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
