不要半听：在持续指令微调中捕获关键部分信息
Don't Half-listen: Capturing Key-part Information in Continual Instruction Tuning
AI总结：
针对持续指令微调中的灾难性遗忘问题，提出基于关键部分信息增益（KPIG）的方法，通过动态重放数据和优化训练目标捕获任务关键信息，并引入P-score和V-score指标，实验证明在已见和保留任务上性能优越。
摘要：
针对大型语言模型（LLMs）的指令微调可以驱动其在特定下游任务中产生与人类目标一致的结果。然而，对LLMs进行持续指令微调（CIT）的过程可能会带来灾难性遗忘（CF）问题，即先前学习的能力会退化。近期的方法试图通过修改模型或重放数据来缓解CF问题，但这些方法可能仅记住指令的表面模式，并在保留任务上感到困惑。在本文中，我们提出了一种基于关键部分信息增益（KPIG）的新型持续指令微调方法。我们的方法计算掩码部分的信息增益，以动态重放数据并优化训练目标，从而使LLMs能够捕获与正确响应相关的任务感知信息，并减轻对指令中通用描述的过拟合。此外，我们提出了两个指标，即P-score和V-score，用于衡量LLMs的泛化能力和指令遵循能力。实验表明，我们的方法在已见任务和保留任务上均取得了优越的性能。
英文摘要：
Instruction tuning for large language models (LLMs) can drive them to produce results consistent with human goals in specific downstream tasks. However, the process of continual instruction tuning (CIT) for LLMs may bring about the catastrophic forgetting (CF) problem, where previously learned abilities are degraded. Recent methods try to alleviate the CF problem by modifying models or replaying data, which may only remember the surface-level pattern of instructions and get confused on held-out tasks. In this paper, we propose a novel continual instruction tuning method based on Key-part Information Gain (KPIG). Our method computes the information gain on masked parts to dynamically replay data and refine the training objective, which enables LLMs to capture task-aware information relevant to the correct response and alleviate overfitting to general descriptions in instructions. In addition, we propose two metrics, P-score and V-score, to measure the generalization and instruction-following abilities of LLMs. Experiments demonstrate our method achieves superior performance on both seen and held-out tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
