基于LLM的算术推理：Prolog生成与置换
Arithmetic Reasoning with LLM: Prolog Generation & Permutation
AI总结：
本文提出用LLM生成Prolog程序解决算术问题，以替代易出错的CoT方法，并在GSM8K基准上超越CoT，同时利用谓词置换进行数据增强以提升训练鲁棒性。
摘要：
使用思维链（CoT）指导大型语言模型（LLMs）解决小学数学问题已展现出巨大成功。然而，CoT方法依赖于LLM生成一系列算术计算，这容易导致级联计算错误。我们假设LLM应专注于从数学问题描述中提取谓词并生成符号公式，以便通过外部代码解释器执行底层计算。我们研究了使用LLM生成Prolog程序来解决数学问题。实验结果表明，在GSM8K基准测试中，我们基于Prolog的算术问题求解方法在三个不同的LLM上均优于CoT生成方法。此外，鉴于Prolog中谓词和符号公式的顺序不敏感性，我们提出对真实谓词进行置换，通过数据增强实现更稳健的LLM训练。
英文摘要：
Instructing large language models (LLMs) to solve elementary school math problems has shown great success using Chain of Thought (CoT). However, the CoT approach relies on an LLM to generate a sequence of arithmetic calculations which can be prone to cascaded calculation errors. We hypothesize that an LLM should focus on extracting predicates and generating symbolic formulas from the math problem description so that the underlying calculation can be done via an external code interpreter. We investigate using LLM to generate Prolog programs to solve mathematical questions. Experimental results show that our Prolog-based arithmetic problem-solving outperforms CoT generation in the GSM8K benchmark across three distinct LLMs. In addition, given the insensitive ordering of predicates and symbolic formulas in Prolog, we propose to permute the ground truth predicates for more robust LLM training via data augmentation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
