arXiv 2505.122502025-05-18cs.CLcs.AI

提取指令微调数据并非需要所有文档

Not All Documents Are What You Need for Extracting Instruction Tuning Data

Department of Computer Science, Beijing Institute of Technology, Beijing, China（北京理工大学计算机科学系）；Sensetime Research, Shenzhen, China（深睿时代研究）；Shanghai Artificial Intelligence Laboratory, Shanghai, China（上海人工智能实验室）；University of Arizona, USA（亚利桑那大学）；Meta, USA（Meta）

Chi Zhang, Huaping Zhong, Hongtao Li, Chengliang Chai, Jiawei Hong, Yuhao Deng, Jiacheng Wang, Tian Tan, Yizhou Yan, Jiantao Qiu, Ye Yuan, Guoren Wang, Conghui He, Lei Cao

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对现有指令微调合成数据多样性不足的问题，提出EQUAL框架，通过文档聚类与多臂老虎机策略迭代筛选文档并提取问答对，在降低计算成本的同时提升了大模型下游任务性能。

摘要：

指令微调可提升大语言模型（LLM）的性能，但它严重依赖高质量训练数据。近期，研究人员利用大语言模型基于种子问答（QA）对合成指令数据。然而，这些合成指令往往缺乏多样性，且与输入种子高度相似，限制了其在真实场景中的适用性。为解决这一问题，我们提出从蕴含丰富多样知识的网络语料中提取指令微调数据。一种朴素的方案是检索特定领域文档并从中提取所有问答对，但这面临两大关键挑战：（1）使用大语言模型提取所有问答对的成本高得令人难以承受；（2）许多提取出的问答对可能与下游任务无关，反而可能降低模型性能。为应对这些问题，我们提出了EQUAL——一种高效且可扩展的数据提取框架，它通过迭代交替进行文档选择与高质量问答对提取，来增强指令微调效果。EQUAL首先基于对比学习得到的嵌入向量对文档语料进行聚类，随后采用多臂老虎机策略高效识别出可能包含有价值问答对的聚类。这种迭代方法在大幅降低计算成本的同时提升了模型性能。在AutoMathText和StackOverflow数据集上针对四项下游任务的实验表明，EQUAL将计算成本降低了5-10倍，并且在LLaMA-3.1-8B和Mistral-7B模型上将准确率提升了2.5个百分点。

英文摘要：

Instruction tuning improves the performance of large language models (LLMs), but it heavily relies on high-quality training data. Recently, LLMs have been used to synthesize instruction data using seed question-answer (QA) pairs. However, these synthesized instructions often lack diversity and tend to be similar to the input seeds, limiting their applicability in real-world scenarios. To address this, we propose extracting instruction tuning data from web corpora that contain rich and diverse knowledge. A naive solution is to retrieve domain-specific documents and extract all QA pairs from them, but this faces two key challenges: (1) extracting all QA pairs using LLMs is prohibitively expensive, and (2) many extracted QA pairs may be irrelevant to the downstream tasks, potentially degrading model performance. To tackle these issues, we introduce EQUAL, an effective and scalable data extraction framework that iteratively alternates between document selection and high-quality QA pair extraction to enhance instruction tuning. EQUAL first clusters the document corpus based on embeddings derived from contrastive learning, then uses a multi-armed bandit strategy to efficiently identify clusters that are likely to contain valuable QA pairs. This iterative approach significantly reduces computational cost while boosting model performance. Experiments on AutoMathText and StackOverflow across four downstream tasks show that EQUAL reduces computational costs by 5-10x and improves accuracy by 2.5 percent on LLaMA-3.1-8B and Mistral-7B

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。