arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2303.05398cs.CLcs.AI
收藏

MathPrompter:使用大语言模型的数学推理

MathPrompter: Mathematical Reasoning using Large Language Models

Microsoft Research(微软研究院)

Shima Imani, Liang Du, Harsh Shrivastava

摘要:

大语言模型(LLM)在解决算术推理任务时性能有限,并且经常提供不正确的答案。与自然语言理解不同,数学问题通常只有一个正确答案,这使得生成精确解决方案的任务对LLM更具挑战性。据我们所知,我们不知道有任何LLM表明他们对自己的反应的信心程度,这加剧了对这些模型的信任赤字,阻碍了它们的采用。为了解决这一缺陷,我们提出了“数学家”,一种技术,提高性能的LLM的算术问题,随着增加的依赖性的预测。Mathemeter使用Zero-shot思想链提示技术生成多个代数表达式或Python函数,以不同的方式解决同一个数学问题,从而提高输出结果的置信度。这与其他基于提示的CoT方法相反,在其他基于提示的CoT方法中,不检查所遵循的中间步骤的有效性。我们的技术改进了使用175 B参数基于GPT的LLM评估的MultiArith数据集($78.7\%\rightarrow92.5\%$)的最新技术。

英文摘要:

Large Language Models (LLMs) have limited performance when solving arithmetic reasoning tasks and often provide incorrect answers. Unlike natural language understanding, math problems typically have a single correct answer, making the task of generating accurate solutions more challenging for LLMs. To the best of our knowledge, we are not aware of any LLMs that indicate their level of confidence in their responses which fuels a trust deficit in these models impeding their adoption. To address this deficiency, we propose `MathPrompter', a technique that improves performance of LLMs on arithmetic problems along with increased reliance in the predictions. MathPrompter uses the Zero-shot chain-of-thought prompting technique to generate multiple Algebraic expressions or Python functions to solve the same math problem in different ways and thereby raise the confidence level in the output results. This is in contrast to other prompt based CoT methods, where there is no check on the validity of the intermediate steps followed. Our technique improves over state-of-the-art on the MultiArith dataset ($78.7\%\rightarrow92.5\%$) evaluated using 175B parameter GPT-based LLM.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑