arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.01560cs.CLcs.AIcs.LG
收藏

OpenMathInstruct-2:借助海量开源指令数据加速数学领域AI发展

OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data

NVIDIA(英伟达)

Shubham Toshniwal, Wei Du, Ivan Moshkov, Branislav Kisacanin, Alexan Ayrapetyan, Igor Gitman

AI总结:

针对LLM数学推理训练数据闭源的问题,通过消融实验总结数据合成规律,构建了规模达1400万对的OpenMathInstruct-2开源数据集,微调后模型在MATH基准上性能提升显著。

摘要:

数学推理始终是大语言模型(LLM)开发中备受关注的核心挑战。然而,由于训练数据难以获取,LLM数学推理领域的多数前沿进展均为闭源。这种数据访问的限制使得研究人员无法明晰数据合成与利用过程中不同选择带来的影响。为打造高质量的数学推理监督微调(SFT)数据集,我们利用最新发布的Llama3.1模型系列开展了细致的消融实验。实验结果表明:(a)解答格式至关重要,过于冗长的解答会损害SFT性能;(b)强教师模型生成的数据,效果优于规模相同的弱学生模型生成的数据;(c)SFT对低质量解答具有鲁棒性,可采用宽松的数据过滤标准;(d)问题多样性对于实现数据规模增益至关重要。基于这些发现,我们构建了OpenMathInstruct-2数据集,包含1400万道问题-解答对(约60万个独立问题),规模约为此前最大的开源数学推理数据集的8倍。使用OpenMathInstruct-2微调Llama-3.1-8B-Base后,其在MATH基准上的表现优于Llama3.1-8B-Instruct,绝对提升达15.9%(从51.9%升至67.8%)。最后,为推动开源领域的发展,我们以商业友好的许可协议开放了代码、微调模型及OpenMathInstruct-2数据集。

英文摘要:

Mathematical reasoning continues to be a critical challenge in large language model (LLM) development with significant interest. However, most of the cutting-edge progress in mathematical reasoning with LLMs has become \emph{closed-source} due to lack of access to training data. This lack of data access limits researchers from understanding the impact of different choices for synthesizing and utilizing the data. With the goal of creating a high-quality finetuning (SFT) dataset for math reasoning, we conduct careful ablation experiments on data synthesis using the recently released \texttt{Llama3.1} family of models. Our experiments show that: (a) solution format matters, with excessively verbose solutions proving detrimental to SFT performance, (b) data generated by a strong teacher outperforms equally-sized data generated by a weak student model, (c) SFT is robust to low-quality solutions, allowing for imprecise data filtering, and (d) question diversity is crucial for achieving data scaling gains. Based on these insights, we create the OpenMathInstruct-2 dataset, which consists of 14M question-solution pairs ($\approx$ 600K unique questions), making it nearly eight times larger than the previous largest open-source math reasoning dataset. Finetuning the \texttt{Llama-3.1-8B-Base} using OpenMathInstruct-2 outperforms \texttt{Llama3.1-8B-Instruct} on MATH by an absolute 15.9\% (51.9\% $\rightarrow$ 67.8\%). Finally, to accelerate the open-source efforts, we release the code, the finetuned models, and the OpenMathInstruct-2 dataset under a commercially permissive license.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑