arXiv 2407.040782024-07-17cs.CLcs.AIcs.LG

DotaMath：借助代码辅助与自我修正的思维分解，用于数学推理

DotaMath: Decomposition of Thought with Code Assistance and Self-correction for Mathematical Reasoning

Alibaba Group（阿里巴巴集团）；University of Science and Technology of China（中国科学技术大学）

Chengpeng Li, Guanting Dong, Mingfeng Xue, Ru Peng, Xiang Wang, Dayiheng Liu

备注：: Work in progress

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出DotaMath模型，通过思维分解、代码辅助与自我修正解决复杂数学推理，构建DotaMathQA数据集训练，在MATH和GSM8K上分别达到64.8%和86.7%的准确率。

摘要：

大型语言模型（LLMs）在处理简单数学问题方面取得了显著进展，但在应对更具挑战性和复杂性的数学任务时仍显不足。本文提出了一系列采用“思维分解、代码辅助与自我修正”策略的LLMs，命名为DotaMath。DotaMath模型通过将复杂数学任务分解为更简单的逻辑子任务，利用代码求解这些子任务，从代码解释器获取细粒度反馈，并进行自我反思与修正，从而解决复杂数学问题。通过标注多样化的交互式工具使用轨迹，并在GSM8K和MATH数据集上应用查询演化，我们生成了一个指令微调数据集DotaMathQA，包含574K个查询-响应对。我们使用模仿学习在DotaMathQA上训练了一系列基础LLMs，得到DotaMath模型，在各种域内和域外基准测试中，与开源LLMs相比，取得了显著性能。值得注意的是，DotaMath-deepseek-7B在竞争性MATH数据集上表现出色，达到64.8%的准确率，在GSM8K上达到86.7%。此外，DotaMath-deepseek-7B在一系列域内和域外基准测试中保持强劲竞争力（平均80.1%）。展望未来，我们预期DotaMath范式将为解决复杂数学问题开辟新途径。我们的代码已公开于https://github.com/ChengpengLi1003/DotaMath。

英文摘要：

Large language models (LLMs) have made impressive progress in handling simple math problems, yet they still struggle with more challenging and complex mathematical tasks. In this paper, we introduce a series of LLMs that employs the Decomposition of thought with code assistance and self-correction for mathematical reasoning, dubbed as DotaMath. DotaMath models tackle complex mathematical tasks by decomposing them into simpler logical subtasks, leveraging code to solve these subtasks, obtaining fine-grained feedback from the code interpreter, and engaging in self-reflection and correction. By annotating diverse interactive tool-use trajectories and employing query evolution on GSM8K and MATH datasets, we generate an instruction fine-tuning dataset called DotaMathQA with 574K query-response pairs. We train a series of base LLMs using imitation learning on DotaMathQA, resulting in DotaMath models that achieve remarkable performance compared to open-source LLMs across various in-domain and out-of-domain benchmarks. Notably, DotaMath-deepseek-7B showcases an outstanding performance of 64.8% on the competitive MATH dataset and 86.7% on GSM8K. Besides, DotaMath-deepseek-7B maintains strong competitiveness on a series of in-domain and out-of-domain benchmarks (Avg. 80.1%). Looking forward, we anticipate that the DotaMath paradigm will open new pathways for addressing intricate mathematical problems. Our code is publicly available at https://github.com/ChengpengLi1003/DotaMath.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。