主动指令微调：通过在提示敏感任务上训练提升跨任务泛化能力
Active Instruction Tuning: Improving Cross-Task Generalization by Training on Prompt Sensitive Tasks
AI总结：
本文提出基于提示不确定性的主动指令微调框架，通过模型在扰动提示上的输出分歧识别信息量大的任务进行选择性训练，在NIV2和Self-Instruct数据集上以更少训练任务实现了更好的分布外泛化，并揭示了模糊任务有助于泛化而困难任务无益的规律。
摘要：
指令微调（IT）通过在大规模多样化任务上使用指令训练大型语言模型（LLMs），取得了令人印象深刻的零样本泛化结果。然而，如何选择新任务以提升IT模型的性能和泛化能力仍然是一个未解决的问题。在所有现有任务上训练因计算需求过高而不切实际，而随机选择任务可能导致次优性能。在这项工作中，我们提出了基于提示不确定性的主动指令微调，这是一个新颖的框架，用于识别信息量大的任务，然后在所选任务上主动微调模型。我们用当前模型在扰动提示上的输出分歧来表示新任务的信息量。我们在NIV2和Self-Instruct数据集上的实验表明，我们的方法在任务选择上始终优于其他基线策略，以更少的训练任务实现了更好的分布外泛化。此外，我们引入了一个任务图谱，根据提示不确定性和预测概率对任务进行分类和诊断。我们发现，在模糊（提示不确定）任务上训练可以提升泛化能力，而在困难（提示确定且低概率）任务上训练没有益处，这凸显了任务选择对于指令微调的重要性。
英文摘要：
Instruction tuning (IT) achieves impressive zero-shot generalization results by training large language models (LLMs) on a massive amount of diverse tasks with instructions. However, how to select new tasks to improve the performance and generalizability of IT models remains an open question. Training on all existing tasks is impractical due to prohibiting computation requirements, and randomly selecting tasks can lead to suboptimal performance. In this work, we propose active instruction tuning based on prompt uncertainty, a novel framework to identify informative tasks, and then actively tune the models on the selected tasks. We represent the informativeness of new tasks with the disagreement of the current model outputs over perturbed prompts. Our experiments on NIV2 and Self-Instruct datasets demonstrate that our method consistently outperforms other baseline strategies for task selection, achieving better out-of-distribution generalization with fewer training tasks. Additionally, we introduce a task map that categorizes and diagnoses tasks based on prompt uncertainty and prediction probability. We discover that training on ambiguous (prompt-uncertain) tasks improves generalization while training on difficult (prompt-certain and low-probability) tasks offers no benefit, underscoring the importance of task selection for instruction tuning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
