arXiv 2510.198422026-03-01cs.AIcs.LG

DAG-Math: 图思维引导的大型语言模型数学推理

DAG-Math: Graph-of-Thought Guided Mathematical Reasoning in LLMs

Department of Statistics, University of Warwick（沃里克大学统计系）；Google DeepMind（谷歌DeepMind）；Department of Applied Mathematics, Hong Kong Polytechnic University（香港理工大学应用数学系）；School of Mathematical Sciences, Institute of Natural Sciences and MOE-LSC, Shanghai Jiao Tong University（上海交通大学数学科学学院）

Yuanhe Zhang, Ilja Kuzborskij, Jason D. Lee, Chenlei Leng, Fanghui Liu

备注：: Accepted by ICLR 2026

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

DAG-Math通过构建图思维引导的数学推理框架，评估LLM的推理能力，揭示了推理忠实度与最终答案准确性的差异。

摘要：

大型语言模型（LLMs）在被提示使用链式思维（CoT）时表现出在数学问题上的强大性能，但尚不清楚这种成功是源于搜索、记忆性程序还是规则一致的推理。为此，我们提出将CoT建模为一种基于规则的随机过程，该过程在有向无环图（DAGs）上进行，其中节点代表中间推导状态，边编码规则应用。在此框架中，我们引入了逻辑接近度，这是一个度量，用于量化模型的CoT轨迹（即LLM的最终输出）与DAG结构的契合程度，提供了超越经典PASS@$k$指标的评估。基于此，我们引入了DAG-MATH CoT格式，并构建了一个基准，引导LLM生成这种格式的CoT轨迹，从而在我们的框架下评估其推理能力。在标准数学推理数据集中，我们的分析揭示了在代表性的LLM家族中，推理忠实度存在统计学显著差异，即使PASS@$k$相近，也凸显出最终答案准确性和规则一致推导之间的差距。我们的框架在自由形式CoT和正式证明系统之间提供了平衡，为LLM推理评估提供了可操作的诊断。我们的基准和代码可在https://github.com/YuanheZ/DAG-MATH上获取。

英文摘要：

Large Language Models (LLMs) demonstrate strong performance on mathematical problems when prompted with Chain-of-Thought (CoT), yet it remains unclear whether this success stems from search, rote procedures, or rule-consistent reasoning. To address this, we propose modeling CoT as a certain rule-based stochastic process over directed acyclic graphs (DAGs), where nodes represent intermediate derivation states and edges encode rule applications. Within this framework, we introduce \textbf{logical closeness}, a metric that quantifies how well a model's CoT trajectory (i.e., the LLM's final output) adheres to the DAG structure, providing evaluation beyond classical PASS@$k$ metrics. Building on this, we introduce the \emph{DAG-MATH} CoT format and construct a benchmark that guides LLMs to generate CoT trajectories in this format, thereby enabling the evaluation of their reasoning ability under our framework. Across standard mathematical reasoning datasets, our analysis uncovers statistically significant differences in reasoning fidelity among representative LLM families-even when PASS@$k$ is comparable-highlighting gaps between final-answer accuracy and rule-consistent derivation. Our framework provides a balance between free-form CoT and formal proofs systems, offering actionable diagnostics for LLMs reasoning evaluation. Our benchmark and code are available at https://github.com/YuanheZ/DAG-MATH.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。