你阅读指令了吗？重新思考任务定义在指令学习中的有效性
Did You Read the Instructions? Rethinking the Effectiveness of Task Definitions in Instruction Learning
AI总结：
本文系统分析指令学习中任务定义的作用，发现输出与标签信息最关键，并提出自动压缩、结构化关键信息和元微调策略，在119个未知任务上提升4.2 Rouge-L。
摘要：
大型语言模型（LLM）在遵循自然语言指令以解决未见过的任务方面表现出色。然而，目前仍不清楚模型是否真正理解任务定义，以及人类编写的定义是否最优。本文系统研究了任务定义在指令学习中的作用。我们首先在人工标注的指导下开展消融分析，以理解任务定义的哪些部分最重要，并发现只有在移除描述任务输出的内容（尤其是标签信息）时，模型性能才会大幅下降。接着，我们提出一种自动算法，将任务定义压缩为最小支持 token 集合，并发现在保持甚至提升模型性能的同时，可以移除 60% 的 token。基于这些结果，我们提出两种策略以帮助模型更好地利用任务指令：（1）以通用结构化格式仅提供任务关键信息；（2）增加一个元微调阶段，帮助模型更好地理解定义。借助这两种策略，我们在 119 个未见过的测试任务上实现了 4.2 Rouge-L 的提升。
英文摘要：
Large language models (LLMs) have shown impressive performance in following natural language instructions to solve unseen tasks. However, it remains unclear whether models truly understand task definitions and whether the human-written definitions are optimal. In this paper, we systematically study the role of task definitions in instruction learning. We first conduct an ablation analysis informed by human annotations to understand which parts of a task definition are most important, and find that model performance only drops substantially when removing contents describing the task output, in particular label information. Next, we propose an automatic algorithm to compress task definitions to a minimal supporting set of tokens, and find that 60\% of tokens can be removed while maintaining or even improving model performance. Based on these results, we propose two strategies to help models better leverage task instructions: (1) providing only key information for tasks in a common structured format, and (2) adding a meta-tuning stage to help the model better understand the definitions. With these two strategies, we achieve a 4.2 Rouge-L improvement over 119 unseen test tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
