arXiv 2503.237142025-08-14cs.CL

使用开放权重大语言模型从人类编写的指令构建指令微调数据集

Building Instruction-Tuning Datasets from Human-Written Instructions with Open-Weight Large Language Models

Department of Computer Science, School of Computing, Institute of Science Tokyo（计算机科学系、计算学院、科学东京研究所）；National Institute of Advanced Industrial Science and Technology（国家先进工业科学与技术研究院）；Research and Development Center for Large Language Models, NII（大型语言模型研究与开发中心、日本科学振兴会）

Youmi Ma, Sakae Mizuki, Kazuki Fujii, Taishi Nakamura, Masanari Ohi, Hinari Shimada, Taihei Shiotani, Koshiro Saito, Koki Maeda, Kakeru Hattori, Takumi Okamoto,… 展开作者

Youmi Ma, Sakae Mizuki, Kazuki Fujii, Taishi Nakamura, Masanari Ohi, Hinari Shimada, Taihei Shiotani, Koshiro Saito, Koki Maeda, Kakeru Hattori, Takumi Okamoto, Shigeki Ishida, Rio Yokota, Hiroya Takamura, Naoaki Okazaki

备注：: COLM 2025; Datasets are available at https://huggingface.co/datasets/tokyotech-llm/lmsys-chat-1m-synth

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本研究通过将人类编写的指令与开放权重LLM生成的回复配对，构建了最先进的指令微调数据集，证明人类指令信号在指令微调中仍具优势，并验证了该方法在日语上的有效性和局限性。

摘要：

指令微调对于使大语言模型（LLMs）能够解决现实世界任务至关重要。先前的工作已经证明了完全由LLMs合成的指令微调数据的有效性，这引发了一个根本性问题：我们是否仍然需要人类起源的信号来进行指令微调？这项工作给出了肯定的答案：我们通过简单地将人类编写的指令与LLM生成的回复配对，构建了源自人类编写指令的最先进的指令微调数据集。在我们的数据集上微调的LLMs始终优于在现有数据集上微调的LLMs。我们的数据构建方法可以轻松适应其他语言；我们为日语构建了数据集，并确认使用我们的数据微调的LLMs达到了最先进的性能。分析表明，在新语言中进行指令微调使LLMs能够遵循指令，而微调后的模型在该语言中表现出明显缺乏文化特定知识。数据集和微调后的模型将公开发布。我们的数据集使用开放权重LLMs合成，在宽松的许可证下公开分发，允许各种用例。

英文摘要：

Instruction tuning is crucial for enabling Large Language Models (LLMs) to solve real-world tasks. Prior work has shown the effectiveness of instruction-tuning data synthesized solely from LLMs, raising a fundamental question: Do we still need human-originated signals for instruction tuning? This work answers the question affirmatively: we build state-of-the-art instruction-tuning datasets sourced from human-written instructions, by simply pairing them with LLM-generated responses. LLMs fine-tuned on our datasets consistently outperform those fine-tuned on existing ones. Our data construction approach can be easily adapted to other languages; we build datasets for Japanese and confirm that LLMs tuned with our data reach state-of-the-art performance. Analyses suggest that instruction-tuning in a new language allows LLMs to follow instructions, while the tuned models exhibit a notable lack of culture-specific knowledge in that language. The datasets and fine-tuned models will be publicly available. Our datasets, synthesized with open-weight LLMs, are openly distributed under permissive licenses, allowing for diverse use cases.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。