arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.17233cs.AI
收藏

草稿与修剪:提升逻辑推理自动形式化可靠性

Draft-and-Prune: Improving the Reliability of Auto-formalization for Logical Reasoning

University of California, Berkeley(加州大学伯克利分校);Microsoft(微软)

Zhiyu Ni, Zheng Liang, Liangcheng Song, Chenrui Cao, Xian Zhang, Alberto Sangiovanni-Vincentelli, Pierluigi Nuzzo

AI总结:

本文提出D&P框架,通过多样性与验证提升自动形式化逻辑推理可靠性,实验显示在多个基准测试中显著优于现有方法,如在AR-LSAT上准确率达78.43%。

摘要:

自动形式化(AF)将自然语言推理问题转换为求解器可执行的程序,使符号求解器能执行可靠的逻辑推断。然而,实际中AF流程易出错:程序可能无法执行或执行但编码错误的语义。尽管先前工作主要通过求解器反馈修复语法错误,但减少语义错误仍是主要瓶颈。本文提出Draft-and-Prune(D&P),一个推理时框架,通过多样性与验证提升基于AF的逻辑推理。D&P首先生成多个自然语言计划和条件程序,然后修剪可执行但矛盾或模糊的形式化,并通过多数投票聚合存活路径的预测。在四个代表性基准(AR-LSAT、ProofWriter、PrOntoQA、LogicalDeduction)上,D&P在无额外监督的情况下显著增强AF推理能力。在AR-LSAT上,D&P在仅AF设置下,使用GPT-4和GPT-4o分别达到78.43%和78.00%的准确率,显著优于最强的AF基线MAD-LOGIC和CLOVER。D&P在其他基准上也接近天花板性能,包括在PrOntoQA和LogicalDeduction上达到100%。

英文摘要:

Auto-formalization (AF) translates natural-language reasoning problems into solver-executable programs, enabling symbolic solvers to perform sound logical deduction. In practice, however, AF pipelines are currently brittle: programs may fail to execute, or execute but encode incorrect semantics. While prior work largely mitigates syntactic failures via repairs based on solver feedback, reducing semantics failures remains a major bottleneck. We propose Draft-and-Prune (D&P), an inference-time framework that improves AF-based logical reasoning via diversity and verification. D&P first drafts multiple natural-language plans and conditions program generation on them. It further prunes executable but contradictory or ambiguous formalizations, and aggregates predictions from surviving paths via majority voting. Across four representative benchmarks (AR-LSAT, ProofWriter, PrOntoQA, LogicalDeduction), D&P substantially strengthens AF-based reasoning without extra supervision. On AR-LSAT, in the AF-only setting, D&P achieves 78.43% accuracy with GPT-4 and 78.00% accuracy with GPT-4o, significantly outperforming the strongest AF baselines MAD-LOGIC and CLOVER. D&P then attains near-ceiling performance on the other benchmarks, including 100% on PrOntoQA and LogicalDeduction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑