超越符号求解:用于大语言模型几何推理的多链式思维投票
Beyond Symbolic Solving: Multi Chain-of-Thought Voting for Geometric Reasoning in Large Language Models
AI总结:
本文提出MARS-GPS方法,通过生成多个并行推理流程并结合Python代码执行,利用熵值进行排序并多阶段投票,提升大语言模型在几何推理中的性能,实验显示其在Geometry3K数据集上达到88.8%的准确率,比现有最佳水平提升11%。
摘要:
几何问题求解(GPS)仍然是增强大语言模型数学推理能力的核心,因为它需要结合图示理解、符号操作和逻辑推理。在现有文献中,研究者主要关注将图示描述与文本字面量同步并解决问题,采用神经、符号或神经符号方法。但这些方法仅解决了图示理解和符号操作的前两个要求,而逻辑推理发展不足。逻辑推理往往局限于单一链式思维(CoT)。为解决现有模型的这一弱点,本文提出MARS-GPS,生成多个并行推理流程并附加Python代码执行以进行数值验证,利用令牌级熵值作为置信度信号进行排序,并通过多阶段投票和自我验证流程汇总答案。实验结果表明,MARS-GPS在8个并行流程下在Geometry3K数据集上达到88.8%的准确率,比先前的最佳水平提高近11%,随着流程数量从1增加到16,准确性持续提升(在消融子集上提升6.0%)。我们提供代码和数据在一个匿名仓库中:https://anonymous.4open.science/r/MARS-GPS-DE55。
英文摘要:
Geometric Problem Solving (GPS) remains at the heart of enhancing mathematical reasoning in large language models because it requires the combination of diagrammatic understanding, symbolic manipulation and logical inference. In existing literature, researchers have chiefly focused on synchronising the diagram descriptions with text literals and solving the problem. In this vein, they have either taken a neural, symbolic or neuro-symbolic approach. But this solves only the first two of the requirements, namely diagrammatic understanding and symbolic manipulation, while leaving logical inference underdeveloped. The logical inference is often limited to one chain-of-thought (CoT). To address this weakness in hitherto existing models, this paper proposes MARS-GPS, that generates multiple parallel reasoning rollouts augmented with Python code execution for numerical verification, ranks them using token-level entropy as a confidence signal, and aggregates answers through a multi-stage voting and self-verification pipeline. Empirical results show that MARS-GPS with 8 parallel rollouts achieves 88.8% on Geometry3K, a nearly +11% improvement over the prior state-of-the-art, with accuracy scaling consistently as the number of rollouts increases from 1 to 16 (+6.0% on ablation subset). We provide our code and data in an anonymous repository: https://anonymous.4open.science/r/MARS-GPS-DE55.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。