arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.05384math.HOcs.AI
收藏

从优秀到卓越:通过工具增强的交错提示提升数学推理能力

From Good to Great: Improving Math Reasoning with Tool-Augmented Interleaf Prompting

Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Hong Kong University of Science and Technology(香港科技大学)

Nuo Chen, Hongguang Li, Baoyuan Wang, Jia Li

AI总结:

本文提出IMP-TIP框架,通过自我提示与工具增强交错提示结合LLM和工具增强LLM的多个解答并交叉检验,在GSM8K-Hard上将准确率从56.0%提升至65.2%,显著增强数学推理能力。

摘要:

本文研究了大语言模型(LLMs)和工具增强的大语言模型在处理复杂数学推理任务中的表现。我们提出了IMP-TIP:通过工具增强的交错提示提升数学推理能力,这是一个结合了大语言模型和工具增强大语言模型各自优势的框架。IMP-TIP遵循“从优秀到卓越”的理念,针对同一道数学题,从大语言模型及其工具增强版本中收集多个潜在解答,然后通过工具增强的交错提示对这些解答进行交叉检验,选择或重新生成最准确的答案。该框架包含两个关键方面:自我提示和工具增强的交错提示(TIP)。前者允许大语言模型自主优化和改进与工具使用相关的初始提示,后者使大语言模型能够通过动态分析问题、交叉检验潜在解答,并以交错方式修正先前的推理提示来得出最终答案。实验分析表明,IMP-TIP在数学推理任务上实现了更强的数学能力,并在准确率和推理多样性方面优于传统大语言模型和工具增强大语言模型。例如,IMP-TIP可以将工具增强的ChatGPT在GSM8K-Hard上的准确率从56.0%提升到65.2%。

英文摘要:

This paper investigates the performance of Large Language Models (LLMs) and Tool-augmented LLMs in tackling complex mathematical reasoning tasks. We introduce IMP-TIP: Improving Math Reasoning with Tool-augmented Interleaf Prompting, a framework that combines the strengths of both LLMs and Tool-augmented LLMs. IMP-TIP follows the ``From Good to Great" concept, collecting multiple potential solutions from both LLMs and their Tool-Augmented counterparts for the same math problem, and then selecting or re-generating the most accurate answer after cross-checking these solutions via tool-augmented interleaf prompting. The framework incorporates two key aspects: self-prompt and tool-augmented interleaf prompting (TIP). The former allows LLMs to autonomously refine and improve an initial prompt related to tool usage, while the latter enables LLMs to derive the final answer by dynamically analyzing the problem, cross-checking potential solutions, and revising previous reasoning hints in an interleaved manner. Experimental analysis shows that IMP-TIP achieves enhanced mathematical capabilities and outperforms traditional LLMs and tool-augmented LLMs in accuracy and reasoning diversity on math reasoning tasks. For instance, IMP-TIP can improve Tool-augmented ChatGPT on GSM8K-Hard from 56.0% to 65.2%.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑