REInstruct：从未标注语料库构建指令数据
REInstruct: Building Instruction Data from Unlabeled Corpus
AI总结：
REInstruct提出一种无需专有模型和人工标注、从未标注语料自动构建指令数据的方法，通过文本筛选与重写提升质量，训练Llama-7b在AlpacaEval上以65.41%胜率超越同类方法。
摘要：
为大型语言模型手动标注指令数据是困难、昂贵且难以扩展的。与此同时，当前自动标注方法通常依赖于从专有大型语言模型中蒸馏合成数据，这不仅限制了指令数据质量的上限，还可能引发版权问题。在本文中，我们提出REInstruct，一种简单且可扩展的方法，能够从未标注语料库自动构建指令数据，而无需重度依赖专有大型语言模型和人工标注。具体而言，REInstruct首先从未标注文本中选取可能包含结构良好、有用且富有洞见内容的子集，然后为这些文本生成指令。为了生成准确且相关的响应以实现有效且稳健的训练，REInstruct进一步提出了一种基于重写的方法来提高生成的指令数据的质量。通过在3k种子数据和REInstruct生成的32k合成数据的组合上训练Llama-7b，微调后的模型在AlpacaEval排行榜上对text-davinci-003取得了65.41%的胜率，优于其他开源、非蒸馏的指令数据构建方法。代码已在\url{https://github.com/cs32963/REInstruct}公开。
英文摘要：
Manually annotating instruction data for large language models is difficult, costly, and hard to scale. Meanwhile, current automatic annotation methods typically rely on distilling synthetic data from proprietary LLMs, which not only limits the upper bound of the quality of the instruction data but also raises potential copyright issues. In this paper, we propose REInstruct, a simple and scalable method to automatically build instruction data from an unlabeled corpus without heavy reliance on proprietary LLMs and human annotation. Specifically, REInstruct first selects a subset of unlabeled texts that potentially contain well-structured helpful and insightful content and then generates instructions for these texts. To generate accurate and relevant responses for effective and robust training, REInstruct further proposes a rewriting-based approach to improve the quality of the generated instruction data. By training Llama-7b on a combination of 3k seed data and 32k synthetic data from REInstruct, fine-tuned model achieves a 65.41\% win rate on AlpacaEval leaderboard against text-davinci-003, outperforming other open-source, non-distilled instruction data construction methods. The code is publicly available at \url{https://github.com/cs32963/REInstruct}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
