MAmmoTH2：从网络扩展指令
MAmmoTH2: Scaling Instructions from the Web
AI总结：
本文提出从网络语料中高效获取千万级指令数据的新范式，构建MAmmoTH2模型，显著提升LLM推理性能，无需人工标注或GPT-4蒸馏。
摘要：
指令微调通过提升数据质量和可扩展性来增强大型语言模型（LLM）的推理能力，其中数据质量和可扩展性是关键因素。大多数指令微调数据来源于人工众包或GPT-4蒸馏。我们提出了一种范式，从预训练网络语料库中高效获取1000万条自然存在的指令数据，以增强LLM的推理能力。我们的方法包括：（1）召回相关文档，（2）提取指令-响应对，（3）使用开源LLM精炼提取的对。在此数据集上微调基础LLM，我们构建了MAmmoTH2模型，显著提升了推理基准的性能。值得注意的是，MAmmoTH2-7B（Mistral）在MATH上的性能从11%提升至36.7%，在GSM8K上从36%提升至68.4%，且未在任何领域内数据上进行训练。进一步在公共指令微调数据集上训练MAmmoTH2，得到MAmmoTH2-Plus，在多个推理和聊天机器人基准上达到了最先进的性能。我们的工作展示了如何在没有昂贵的人工标注或GPT-4蒸馏的情况下，获取大规模、高质量的指令数据，为构建更好的指令微调数据提供了新范式。
英文摘要：
Instruction tuning improves the reasoning abilities of large language models (LLMs), with data quality and scalability being the crucial factors. Most instruction tuning data come from human crowd-sourcing or GPT-4 distillation. We propose a paradigm to efficiently harvest 10 million naturally existing instruction data from the pre-training web corpus to enhance LLM reasoning. Our approach involves (1) recalling relevant documents, (2) extracting instruction-response pairs, and (3) refining the extracted pairs using open-source LLMs. Fine-tuning base LLMs on this dataset, we build MAmmoTH2 models, which significantly boost performance on reasoning benchmarks. Notably, MAmmoTH2-7B's (Mistral) performance increases from 11% to 36.7% on MATH and from 36% to 68.4% on GSM8K without training on any in-domain data. Further training MAmmoTH2 on public instruction tuning datasets yields MAmmoTH2-Plus, achieving state-of-the-art performance on several reasoning and chatbot benchmarks. Our work demonstrates how to harvest large-scale, high-quality instruction data without costly human annotation or GPT-4 distillation, providing a new paradigm for building better instruction tuning data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
