arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.21044cs.AI
收藏

LogicGraph : 通过神经符号生成与验证进行多路径逻辑推理的基准测试

LogicGraph : Benchmarking Multi-Path Logical Reasoning via Neuro-Symbolic Generation and Verification

School of Computer Science and Technology, Xi’an Jiaotong University（西安交通大学计算机科学与技术学院）；Department of Electrical Engineering, Stanford University（斯坦福大学电气工程系）；School of Computer Science and Technology, Tiangong University（天津大学计算机科学与技术学院）；Ministry of Education Key Laboratory of Intelligent Networks and Network Security, China（中国教育部长智网络与网络安全重点实验室）；Shaanxi Province Key Laboratory of Big Data Knowledge Engineering, China（陕西省大数据知识工程重点实验室）

Yanrui Wu, Lingling Zhang, Xinyu Zhang, Jiayu Chang, Pengyu Li, Xu Jiang, Jingtao Hu, Jun Liu

备注：
24 pages, 17 figures
发表信息：
In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP 2026), pp. 15931-15959, 2026
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

LogicGraph通过神经符号生成与验证方法，系统评估多路径逻辑推理能力，揭示模型在复杂推理中探索多样路径的不足。

摘要：

对大型语言模型（LLMs）的评估主要强调收敛性逻辑推理，其成功定义为生成单一正确的证明。然而，许多现实世界推理问题允许多个有效的推导，要求模型探索多样化的逻辑路径，而不是局限于一条路线。为解决这一限制，我们引入LogicGraph，这是首个旨在系统评估多路径逻辑推理的基准，通过神经符号框架构建，该框架利用反向逻辑生成和语义实例化。该流程产生由高深度多路径推理和内在逻辑干扰正式化的求解器验证推理问题，每个实例都关联着一组完整的最小证明。我们进一步提出一种无参考评估框架，以严格评估模型在收敛和发散模式中的性能。在最先进的语言模型上的实验揭示了共同的限制：模型倾向于早期选择单一路线并无法探索替代方案，且随着推理深度的增加，覆盖差距显著扩大。LogicGraph揭示了这一发散差距，并提供了可操作的见解，以推动未来改进。我们的代码和数据将在https://github.com/kkkkarry/LogicGraph上发布。

英文摘要：

Evaluations of large language models (LLMs) primarily emphasize convergent logical reasoning, where success is defined by producing a single correct proof. However, many real-world reasoning problems admit multiple valid derivations, requiring models to explore diverse logical paths rather than committing to one route. To address this limitation, we introduce LogicGraph, the first benchmark aimed to systematically evaluate multi-path logical reasoning, constructed via a neuro-symbolic framework that leverages backward logic generation and semantic instantiation. This pipeline yields solver-verified reasoning problems formalized by high-depth multi-path reasoning and inherent logical distractions, where each instance is associated with an exhaustive set of minimal proofs. We further propose a reference-free evaluation framework to rigorously assess model performance in both convergent and divergent regimes. Experiments on state-of-the-art language models reveal a common limitation: models tend to commit early to a single route and fail to explore alternatives, and the coverage gap grows substantially with reasoning depth. LogicGraph exposes this divergence gap and provides actionable insights to motivate future improvements. Our code and data will be released at https://github.com/kkkkarry/LogicGraph.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑