大型语言模型在算术任务中表现如何？
How well do Large Language Models perform in Arithmetic tasks?
AI总结：
本研究提出了算术数据集 MATH 401，用于评估 GPT-4、ChatGPT 等大型语言模型在各类算术表达式上的计算能力，填补了该领域评测空白并进行了详细分析。
摘要：
大型语言模型已展现出包括思维链在内的能力，能够逐步解答数学应用题。解答数学应用题不仅需要通过思维链拆解问题的能力，还需要为每一步正确计算算术表达式。据我们所知，目前尚无工作专注于评估大型语言模型的算术能力。在本研究中，我们提出了一个算术数据集 MATH 401，以测试包括 GPT-4、ChatGPT、InstrctGPT、Galactica 和 LLaMA 在内的最新大型语言模型处理各种算术表达式的能力，并对大型语言模型的能力进行了详细分析。MATH 401 和评估代码已发布在 https://github.com/GanjinZero/math401-llm。
英文摘要：
Large language models have emerged abilities including chain-of-thought to answer math word problems step by step. Solving math word problems not only requires abilities to disassemble problems via chain-of-thought but also needs to calculate arithmetic expressions correctly for each step. To the best of our knowledge, there is no work to focus on evaluating the arithmetic ability of large language models. In this work, we propose an arithmetic dataset MATH 401 to test the latest large language models including GPT-4, ChatGPT, InstrctGPT, Galactica, and LLaMA with various arithmetic expressions and provide a detailed analysis of the ability of large language models. MATH 401 and evaluation codes are released at \url{https://github.com/GanjinZero/math401-llm}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
