arXiv 2311.098022023-11cs.AIcs.CL

神经符号集成带来因果且可靠的推理证明

Neuro-Symbolic Integration Brings Causal and Reliable Reasoning Proofs

The Chinese University of Hong Kong（香港中文大学）；DAMO Academy, Alibaba Group（阿里巴巴达摩院）；Shanda AI Research Institute（盛大人工智能研究院）

Sen Yang, Xin Li, Leyang Cui, Lidong Bing, Wai Lam

备注：: To appear in Findings of NAACL2025

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对LLM复杂推理中准确性与可解释性的权衡，提出将Prolog搜索日志转为可读证明的神经符号框架，在逻辑与算术数据集上显著提升答案与证明准确性。

摘要：

针对大型语言模型（LLMs）的复杂推理，现有研究主要沿两条路线展开。一条路线通过提示LLMs采用各种推理结构，其结构化输出可自然视为中间推理步骤。另一条路线采用无LLM的声明式求解器执行推理任务，虽然推理精度更高，但由于求解器的黑盒特性而缺乏可解释性。为解决答案准确性与可解释性之间的权衡，我们提出了对后一路线的简单扩展。具体而言，我们展示了Prolog解释器生成的中间搜索日志可以被访问并解释为人类可读的推理证明。只要LLMs能将问题描述正确翻译为Prolog表示，相应的推理证明就能确保因果性和可靠性。在两个逻辑推理数据集和一个算术推理数据集上，我们的框架在答案准确性和推理证明准确性两方面均取得了显著提升。我们的代码已发布在https://github.com/DAMO-NLP-SG/CaRing。

英文摘要：

Two lines of approaches are adopted for complex reasoning with LLMs. One line of work prompts LLMs with various reasoning structures, while the structural outputs can be naturally regarded as intermediate reasoning steps. Another line of work adopt LLM-free declarative solvers to do the reasoning task, rendering higher reasoning accuracy but lacking interpretability due to the black-box nature of the solvers. Aiming to resolve the trade-off between answer accuracy and interpretability, we present a simple extension to the latter line of work. Specifically, we showcase that the intermediate search logs generated by Prolog interpreters can be accessed and interpreted into human-readable reasoning proofs. As long as LLMs correctly translate problem descriptions into Prolog representations, the corresponding reasoning proofs are ensured to be causal and reliable. On two logical reasoning and one arithmetic reasoning datasets, our framework obtains significant improvements in terms of both answer accuracy and reasoning proof accuracy. Our code is released at https://github.com/DAMO-NLP-SG/CaRing

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

AI总结：

摘要：

英文摘要：

相关论文