arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.17331cs.AIcs.LO
收藏

定理证明器需要定理证明智能体吗?mini-Lean-agent:Lean的编码智能体基线

Do Theorem Provers Need Theorem-Proving Agents? mini-Lean-agent: A Coding-Agent Baseline for Lean

Princeton Language and Intelligence(普林斯顿语言与智能中心);Princeton University(普林斯顿大学);Rutgers University(罗格斯大学)

Tomas Ortega, Simon Park, Ayush Khaitan, Liam Fowl, Alex Kontorovich, Sanjeev Arora

AI总结:

该研究针对Lean定理证明任务推出极简智能体mini-Lean-agent,对比实验显示其在Putnam 2025基准上性能优于部分现有智能体,可作为定理证明基础设施评估的标准基线。

摘要:

通用编码智能体采用标准的Shell和文件交互方式,而形式化定理证明智能体往往依赖专门的、临时搭建的工具,这使得难以确定定理证明智能体的哪些组件能带来基准性能提升:是改进的基础模型、专门的框架、特定工具,还是它们的组合。我们推出mini-Lean-agent,这是一款专为Lean任务设计的极简智能体,其唯一对模型可见的工具是bash。模型必须使用bash来检查和编辑仓库、搜索本地库、调用Lean以及管理自身的中间工作,同时由一个独立的可信评估器验证最终成果。我们在Putnam 2025基准上将mini-Lean-agent与AxProverBase和Numina-Lean-Agent进行了对比。以Gemini 3.8 Flash作为基础模型的实验显示,在每个问题2美元的成本限制下,mini-Lean-agent解决了12个问题中的7个,而AxProverBase解决了4个;以Astra作为基础模型时,mini-Lean-agent解决了全部12个问题,平均每个问题成本为1.28美元。这表明,配备极简框架的通用编码智能体已能在Lean基准上取得出色性能,我们的结果推动将mini-Lean-agent作为评估定理证明基础设施的标准基线。

英文摘要:

General-purpose coding agents use standard shell and file interactions, yet formal theorem-proving agents often rely on specialized, ad-hoc tooling. This makes it difficult to determine which components of theorem-proving agents provide benchmark gains: an improved base model, a specialized harness, a specific tool, or a combination of any of them. We introduce mini-Lean-agent, a deliberately minimal agent for Lean tasks whose only model-visible tool is bash. The model must use bash to inspect and edit the repository, search local libraries, invoke Lean, and manage its own intermediate work, while a separate trusted evaluator verifies the final artifact. We compare mini-Lean-agent against AxProverBase and Numina-Lean-Agent on the Putnam 2025 benchmark. Experiments with Gemini 3.8 Flash as the base model show that mini-Lean-agent solves 7/12 problems at \$2 per problem limit, compared with AxProverBase's 4/12. With Astra as the base model, mini-Lean-agent solves 12/12 problems at an average cost of \$1.28 per problem. This suggests that a generic coding agent with a minimal harness can already achieve strong performance on Lean benchmarks. Our results motivate using mini-Lean-agent as a standard baseline when evaluating theorem-proving infrastructure.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑