arXiv 2404.010672024-04cs.CL

探索数学推理中有影响力数据的奥秘

Exploring the Mystery of Influential Data for Mathematical Reasoning

Institute of Data and Information（数据与信息研究所）；Tsinghua University（清华大学）；Microsoft（微软公司）

Xinzhe Ni, Yeyun Gong, Zhibin Gou, Yelong Shen, Yujiu Yang, Nan Duan, Weizhu Chen

备注：: Accepted by COLM 2024

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对数学推理微调的数据选择与组成问题，提出QaDS质量感知多样化选择策略，并构建OpenMathMix，使7B模型在MATH上达到48.8%的最优准确率。

摘要：

为下游任务选择有影响力的数据进行微调，是性能和计算效率的关键因素。近期研究表明，仅使用有限数据进行训练，即可在通用任务上展现出卓越性能。然而，其在数学推理任务上的可行性尚未得到验证。进一步而言，数学推理存在两个开放性问题：如何选择有影响力的数据，以及什么是有影响力的数据组成。针对前者，我们提出了一种适用于数学推理的质量感知多样化选择（Quality-aware Diverse Selection，QaDS）策略。与其他选择策略的比较验证了QaDS的优越性。针对后者，我们首先扩大设置范围，并探索有影响力的数据组成。我们进行了一系列实验，并强调：扩大推理数据规模，以及使用由QaDS选择的通用数据进行训练是有帮助的。随后，我们将最优混合数据定义为OpenMathMix，这是一种由QaDS选择开源数据构成的有影响力的数据混合物。借助OpenMathMix，我们使用7B基础模型在MATH上取得了48.8%准确率的当前最优结果。此外，我们展示了QaDS在创建具有不同选择比例的高效微调混合物方面的用途，并分析了大量开源数据集的质量，这可作为未来数学推理任务研究的参考。

英文摘要：

Selecting influential data for fine-tuning on downstream tasks is a key factor for both performance and computation efficiency. Recent works have shown that training with only limited data can show a superior performance on general tasks. However, the feasibility on mathematical reasoning tasks has not been validated. To go further, there exist two open questions for mathematical reasoning: how to select influential data and what is an influential data composition. For the former one, we propose a Quality-aware Diverse Selection (QaDS) strategy adaptable for mathematical reasoning. A comparison with other selection strategies validates the superiority of QaDS. For the latter one, we first enlarge our setting and explore the influential data composition. We conduct a series of experiments and highlight: scaling up reasoning data, and training with general data selected by QaDS is helpful. Then, we define our optimal mixture as OpenMathMix, an influential data mixture with open-source data selected by QaDS. With OpenMathMix, we achieve a state-of-the-art 48.8% accuracy on MATH with 7B base model. Additionally, we showcase the use of QaDS in creating efficient fine-tuning mixtures with various selection ratios, and analyze the quality of a wide range of open-source datasets, which can perform as a reference for future works on mathematical reasoning tasks.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。