Self-Instruct：利用自生成指令对齐语言模型
Self-Instruct: Aligning Language Models with Self-Generated Instructions
AI总结：
本文提出Self-Instruct框架，通过让语言模型自生成指令数据进行自举微调，在几乎无需人工标注的情况下显著提升指令遵循能力，并在Super-NaturalInstructions上接近InstructGPT-001的性能。
摘要：
大型“指令微调”语言模型（即经过微调以响应指令的模型）已展现出对新任务进行零样本泛化的卓越能力。然而，它们严重依赖人工编写的指令数据，而这些数据通常在数量、多样性和创造性方面有限，从而阻碍了微调后模型的通用性。我们提出了Self-Instruct框架，通过从语言模型自身的生成结果中进行自举，提升预训练语言模型的指令遵循能力。我们的流程从语言模型生成指令、输入和输出样本，然后过滤掉无效或相似的样本，再将其用于微调原始模型。将我们的方法应用于原始GPT3，我们在Super-NaturalInstructions上展示了相较原始模型33%的绝对提升，与使用私有用户数据和人工标注训练的InstructGPT-001的性能相当。为了进一步评估，我们整理了一组由专家为新颖任务编写的指令，并通过人工评估表明，使用Self-Instruct微调GPT3大幅优于使用现有公开指令数据集，与InstructGPT-001之间仅剩5%的绝对差距。Self-Instruct为对齐预训练语言模型与指令提供了一种几乎无需标注的方法，我们发布了大规模合成数据集以促进未来关于指令微调的研究。我们的代码和数据可在https://github.com/yizhongw/self-instruct获取。
英文摘要：
Large "instruction-tuned" language models (i.e., finetuned to respond to instructions) have demonstrated a remarkable ability to generalize zero-shot to new tasks. Nevertheless, they depend heavily on human-written instruction data that is often limited in quantity, diversity, and creativity, therefore hindering the generality of the tuned model. We introduce Self-Instruct, a framework for improving the instruction-following capabilities of pretrained language models by bootstrapping off their own generations. Our pipeline generates instructions, input, and output samples from a language model, then filters invalid or similar ones before using them to finetune the original model. Applying our method to the vanilla GPT3, we demonstrate a 33% absolute improvement over the original model on Super-NaturalInstructions, on par with the performance of InstructGPT-001, which was trained with private user data and human annotations. For further evaluation, we curate a set of expert-written instructions for novel tasks, and show through human evaluation that tuning GPT3 with Self-Instruct outperforms using existing public instruction datasets by a large margin, leaving only a 5% absolute gap behind InstructGPT-001. Self-Instruct provides an almost annotation-free method for aligning pre-trained language models with instructions, and we release our large synthetic dataset to facilitate future studies on instruction tuning. Our code and data are available at https://github.com/yizhongw/self-instruct.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
