时机至关重要：数据排列影响指令微调中的零样本泛化
The Right Time Matters: Data Arrangement Affects Zero-Shot Generalization in Instruction Tuning
AI总结：
本文从数据排列角度研究指令微调中的零样本泛化机制，证明泛化发生在训练早期且损失可作指标，并提出以测试为中心的多轮排列框架来提升持续学习效果。
摘要：
理解对齐技术始于理解指令微调所带来的零样本泛化，但对其机制的理解仍然很少。现有研究大多局限于任务层面，未考虑到任务是人为定义的，并且对于大语言模型而言，任务仅由token和表示组成。为弥合这一差距，我们从数据本身的角度研究零样本泛化。我们首先证明零样本泛化在指令微调过程中很早就出现，损失值可作为一个稳定的指标。接着，我们从相似性和粒度的角度研究训练数据排列，证实某些训练样本的暴露时机可能会极大促进在未见任务上的泛化。最后，我们提出一个更有依据的训练数据排列框架——Test-centric Multi-turn Arrangement（以测试为中心的多轮排列），并展示其在促进持续学习和进一步降低损失方面的有效性。我们首次表明，指令微调期间的零样本泛化是训练数据与测试数据在实例层面基于相似性的一种泛化形式。我们的代码已发布在 https://github.com/thunlp/Dynamics-of-Zero-Shot-Generalization。
英文摘要：
Understanding alignment techniques begins with comprehending zero-shot generalization brought by instruction tuning, but little of the mechanism has been understood. Existing work has largely been confined to the task level, without considering that tasks are artificially defined and, to LLMs, merely consist of tokens and representations. To bridge this gap, we investigate zero-shot generalization from the perspective of the data itself. We first demonstrate that zero-shot generalization happens very early during instruction tuning, with loss serving as a stable indicator. Next, we investigate training data arrangement through similarity and granularity perspectives, confirming that the timing of exposure to certain training examples may greatly facilitate generalization on unseen tasks. Finally, we propose a more grounded training data arrangement framework, Test-centric Multi-turn Arrangement, and show its effectiveness in promoting continual learning and further loss reduction. For the first time, we show that zero-shot generalization during instruction tuning is a form of similarity-based generalization between training and test data at the instance level. Our code is released at https://github.com/thunlp/Dynamics-of-Zero-Shot-Generalization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
