arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.12148cs.AI
收藏

使用大语言模型进行数学推理的多工具集成应用

Multi-tool Integration Application for Math Reasoning Using Large Language Model

China Telecom Shanghai Company(中国电信上海公司);Stanford University(斯坦福大学)

Zhihua Duan, Jialin Wang

AI总结:

本文提出一种结合大语言模型的多工具数学推理框架,通过集成Math Tool、Code Tool、CoT Tool和self consistency工具实现协同推理,在NumGLUE Task 4上准确率达89.09%,显著优于现有基线。

摘要:

数学推理是人工智能领域的重要研究方向。本文提出了一种用于数学推理的新型多工具应用框架,旨在通过利用大语言模型(LLMs)和多个外部工具的协同效应来实现更全面、准确的数学推理。首先,在推理过程中通过使用 Math Tool 与 LLM 交互,执行基础数学计算。其次,Code Tool 能够生成符合语法规则的代码片段并执行,为复杂数学问题提供支持。然后,通过 CoT Tool 的迭代推理,增强了数学推理的逻辑连贯性和准确性。最后,通过使用 self consistency 工具基于不同参数选择最终答案,提升了推理的一致性和可靠性。通过这些工具的协同效应,该框架在数学推理任务中取得了显著的性能提升。我们在 NumGLUE Task 4 测试集上进行了实验,该测试集包含 220 道数学推理填空题。实验结果表明,基于 Math Tool、Code Tool 和 CoT Tool,在 Task 4 任务中,我们的方法达到了 89.09 的准确率,与 GPT3+FewShot 基线相比,Few Shot+ERNIE-4.0+self consistency 提升了 49.09%,与 Fine tuning 基线相比,Few Shot+ERNIE-4.0+self consistency 提升了 52.29%。

英文摘要:

Mathematical reasoning is an important research direction in the field of artificial intelligence. This article proposes a novel multi tool application framework for mathematical reasoning, aiming to achieve more comprehensive and accurate mathematical reasoning by utilizing the collaborative effect of large language models (LLMs) and multiple external tools. Firstly, use a Math Tool to perform basic mathematical calculations during the inference process through interaction with LLM. Secondly, Code Tool can generate code fragments that comply with syntax rules and execute them, providing support for complex mathematical problems. Then, through the iterative reasoning of the CoT Tool, the logical coherence and accuracy of mathematical reasoning are enhanced. Ultimately, by using self consistency tools to select the final answer based on different parameters, the consistency and reliability of reasoning are improved. Through the synergistic effect of these tools, the framework has achieved significant performance improvement in mathematical reasoning tasks. We conducted experiments on the NumGLUE Task 4 test set, which includes 220 mathematical reasoning fill in the blank questions. The experimental results showed that, based on Math Tool, Code Tool, and CoT Tool, in Task 4 task,our method achieved an accuracy of 89.09,compared with the GPT3+FewShot baseline, Few Shot+ERNIE-4.0+self consistency improved by 49.09%, and compared with fine-tuning the Fine tuning baseline, Few Shot+ERNIE-4.0+self consistency improved by 52.29%

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑