arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.18310cs.LGcs.AI
收藏

ProofAug:通过细粒度证明结构分析实现高效神经定理证明

ProofAug: Efficient Neural Theorem Proving via Fine-grained Proof Structure Analysis

Tsinghua University(清华大学);Huawei Noah’s Ark Lab(华为诺亚方舟实验室);Shanghai Qi Zhi Institute(上海期智研究院)

Haoxiong Liu, Jiacheng Sun, Zhenguo Li, Andrew C Yao

AI总结:

ProofAug 通过对模型生成的证明方案进行细粒度结构分析,在多种粒度上为 LLMs 集成自动化方法,并作为即插即用模块与树搜索算法结合,在 miniF2F 上显著提升了 Isabelle 和 Lean 4 中的神经定理证明效率与准确率。

摘要:

深度学习模型与传统自动化工具(如证明助手的内置策略和现成的自动定理证明器)之间的协同作用,在开发稳健且高效的神经定理证明器(NTPs)中起着至关重要的作用。然而,对于使用大语言模型(LLMs)进行证明合成,以往的工作要么仅在模型显式调用时才应用自动化工具,要么只在单一粒度级别上应用,未能充分发挥其能力。为解决这一问题,我们提出了 ProofAug,该流程通过对模型生成的证明方案进行细粒度结构分析,使 LLMs 能够在多种粒度上配备自动化方法。ProofAug 还可作为一个通用的即插即用模块,与任意树搜索算法无缝集成,使我们能够构建一个高效的递归证明(ERP)模块以进一步提升性能。我们的方法在 miniF2F 基准上使用开源 deepseek-math-7b-base 模型和 Isabelle 证明助手进行了验证。值得注意的是,通过额外采用混合提示策略,我们在数据集整理后达到了 66.0% 的累积通过率(原始版本为 61.9%),每个问题对模型进行 2100 次查询(相比之下,Isabelle 中此前的 SOTA 方法 Subgoal-XL 每个问题使用 16384 次查询仅达到 56.1%)。我们还实现了 ProofAug 的 Lean 4 版本,该版本可将 Kimina-Prover-Preview-Distill-1.5B 在 miniF2F-test 上的 pass@1 性能从 44.3% 提升至 50.4%。我们的代码可在 https://github.com/haoxiongliu/ProofAug 获取。

英文摘要:

The synergy between deep learning models and traditional automation tools, such as built-in tactics of the proof assistant and off-the-shelf automated theorem provers, plays a crucial role in developing robust and efficient neural theorem provers(NTPs). However, for proof synthesis with LLMs, previous work applies automation tools either only when explicitly invoked by the model or at a single granularity level, failing to fully exploit their power. To solve this issue, we propose ProofAug, a procedure that equips LLMs with automation methods at various granularities through fine-grained structure analysis of model-generated proof proposals. ProofAug also serves as a versatile plug-and-play module that seamlessly integrates with any tree-search algorithm, enabling our construction of an efficient recursive proving (ERP) module to further enhance performance. The superiority of our method is validated on the miniF2F benchmark using the open-source deepseek-math-7b-base model and the Isabelle proof assistant. Notably, by additionally employing a mixed prompting strategy, we achieve a cumulative pass rate of 66.0% after curation of the dataset (61.9% for the original version) with 2100 queries to the model per problem (In contrast, the previous SOTA in Isabelle, Subgoal-XL, only achieves 56.1% using 16384 queries per problem). We also implement a Lean 4 version of ProofAug that can improve the pass@1 performance of Kimina-Prover-Preview-Distill-1.5B from 44.3% to 50.4% on miniF2F-test. Our code is available at https://github.com/haoxiongliu/ProofAug.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑