arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.10176cs.CLcs.AIcs.LG
收藏

OpenMathInstruct-1:一个包含180万条数据的数学指令微调数据集

OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset

NVIDIA(英伟达)

Shubham Toshniwal, Ivan Moshkov, Sean Narenthiran, Daria Gitman, Fei Jia, Igor Gitman

AI总结:

针对现有数学指令微调数据集依赖闭源受限模型生成的问题,研究者结合开源模型进展与新型提示技术,构建了含180万对数据的开源数据集OpenMathInstruct-1,训练出的70B模型数学推理性能比肩顶级GPT蒸馏模型。

摘要:

近期研究表明,合成生成的数据集在训练大语言模型(LLM)、尤其是帮助模型获得特定技能方面具有巨大潜力。目前的大规模数学指令微调数据集,如MetaMathQA(Yu等人,2024)和MAmmoTH(Yue等人,2024),均采用具有商业限制许可的闭源大语言模型的输出来构建。限制开源大语言模型用于此类数据生成流程的一个关键原因,是GPT-4等顶级闭源大语言模型与顶级开源大语言模型之间在数学能力上存在巨大差距。\n基于开源大语言模型的最新进展、我们提出的新型提示词设计以及一定的暴力式规模扩展,我们构建了OpenMathInstruct-1——一个包含180万道题目-解答对的数学指令微调数据集。该数据集通过使用近期发布、许可宽松的Mixtral模型,为GSM8K和MATH这两个主流数学推理基准合成代码解释器解答而构建。我们在OpenMathInstruct-1的子集上训练得到的最优模型OpenMath-CodeLlama-70B,在GSM8K上取得84.6%的得分,在MATH上取得50.7%的得分,性能可与最优的GPT蒸馏模型相媲美。我们以商业宽松许可发布了代码、模型以及OpenMathInstruct-1数据集。

英文摘要:

Recent work has shown the immense potential of synthetically generated datasets for training large language models (LLMs), especially for acquiring targeted skills. Current large-scale math instruction tuning datasets such as MetaMathQA (Yu et al., 2024) and MAmmoTH (Yue et al., 2024) are constructed using outputs from closed-source LLMs with commercially restrictive licenses. A key reason limiting the use of open-source LLMs in these data generation pipelines has been the wide gap between the mathematical skills of the best closed-source LLMs, such as GPT-4, and the best open-source LLMs. Building on the recent progress in open-source LLMs, our proposed prompting novelty, and some brute-force scaling, we construct OpenMathInstruct-1, a math instruction tuning dataset with 1.8M problem-solution pairs. The dataset is constructed by synthesizing code-interpreter solutions for GSM8K and MATH, two popular math reasoning benchmarks, using the recently released and permissively licensed Mixtral model. Our best model, OpenMath-CodeLlama-70B, trained on a subset of OpenMathInstruct-1, achieves a score of 84.6% on GSM8K and 50.7% on MATH, which is competitive with the best gpt-distilled models. We release our code, models, and the OpenMathInstruct-1 dataset under a commercially permissive license.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑