探索指令微调中的格式一致性
Exploring Format Consistency for Instruction Tuning
AI总结：
本文提出统一指令微调（UIT）框架，通过OpenAI API实现跨数据集格式转换，证明格式一致性必要性，提升T5-LM-xl泛化性能，并基于困惑度去噪及GPT-J离线模型降低成本。
摘要：
指令微调已成为增强大型语言模型遵循人类指令的一种有前途的方法。研究表明，增加训练数据中指令的多样性和数量可以持续提升泛化性能，这促进了近期收集各种指令并将现有指令微调数据集整合为更大集合的努力。然而，不同的用户有其独特的表达指令的方式，并且不同数据集在指令风格和格式上通常存在差异，即格式不一致。在这项工作中，我们提出了一个名为统一指令微调（UIT）的框架，该框架调用OpenAI API在PromptSource、FLAN和CrossFit等不同指令微调数据集之间进行自动格式转换。通过该框架，我们（1）证明了在指令微调中保持格式一致性的必要性；（2）提升了T5-LM-xl在未见指令上的泛化性能；（3）提供了一种基于困惑度的去噪方法，以减少自动格式转换的噪声，使UIT框架更加实用，并基于GPT-J提供了一个较小的离线模型，实现了与OpenAI API相当的格式转换能力，以降低实际成本。我们计划进一步分析目标格式的变化及其他影响。
英文摘要：
Instruction tuning has emerged as a promising approach to enhancing large language models in following human instructions. It is shown that increasing the diversity and number of instructions in the training data can consistently enhance generalization performance, which facilitates a recent endeavor to collect various instructions and integrate existing instruction tuning datasets into larger collections. However, different users have their unique ways of expressing instructions, and there often exist variations across different datasets in the instruction styles and formats, i.e., format inconsistency. In this work, we propose a framework named Unified Instruction Tuning (UIT), which calls OpenAI APIs for automatic format transfer among different instruction tuning datasets such as PromptSource, FLAN and CrossFit. With the framework, we (1) demonstrate the necessity of maintaining format consistency in instruction tuning; (2) improve the generalization performance on unseen instructions on T5-LM-xl; (3) provide a novel perplexity-based denoising method to reduce the noise of automatic format transfer to make the UIT framework more practical and a smaller offline model based on GPT-J that achieves comparable format transfer capability to OpenAI APIs to reduce costs in practice. Further analysis regarding variations of targeted formats and other effects is intended.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
