arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.21065cs.AIcs.CL
收藏

LongCat-Flash-Prover:通过集成工具的强化学习推进原生形式推理

LongCat-Flash-Prover: Advancing Native Formal Reasoning via Agentic Tool-Integrated Reinforcement Learning

Meituan(美团)
Jianing Wang, Jianfei Zhang, Qi Guo, Linsen Guo, Rumei Li, Chao Zhang, Chong Peng, Cunguang Wang, Dengchang Zhao, Jiarong Shi, Jingang Wang, Liulin Feng, Mengxi… 展开作者

Jianing Wang, Jianfei Zhang, Qi Guo, Linsen Guo, Rumei Li, Chao Zhang, Chong Peng, Cunguang Wang, Dengchang Zhao, Jiarong Shi, Jingang Wang, Liulin Feng, Mengxia Shen, Qi Li, Shengnan An, Shun Wang, Wei Shi, Xiangyu Xi, Xiaoyu Li, Xuezhi Cao, Yi Lu, Yunke Zhao, Zhengyu Chen, Zhimin Lin, Wei Wang, Peng Pei, Xunliang Cai

AI总结:

本文提出LongCat-Flash-Prover,一个5600亿参数的混合专家模型,通过集成工具的强化学习提升Lean4中的原生形式推理能力,实现了自动形式化、草图生成和证明三大能力,并通过混合专家迭代框架和分层重要性采样策略提升训练效率。

摘要:

我们介绍了LongCat-Flash-Prover,一个旗舰5600亿参数的混合专家(MoE)模型,通过集成工具的强化学习(TIR)推动Lean4中的原生形式推理。我们将原生形式推理任务分解为三个独立的形式能力,即自动形式化、草图生成和证明。为促进这些能力,我们提出混合专家迭代框架以扩展高质量任务轨迹,包括基于给定非正式问题生成形式化陈述、直接从陈述生成完整证明,或生成引理式草图。在代理强化学习中,我们提出分层重要性采样策略优化(HisPO)算法,旨在稳定MoE模型在长horizon任务上的训练。它采用梯度遮蔽策略,考虑策略陈旧性和内在训练-推理引擎差异,同时在序列和token层面。此外,我们还整合了定理一致性和合法性检测机制以消除奖励黑客问题。大量评估表明,我们的LongCat-Flash-Prover在自动形式化和定理证明方面为开放权重模型设定了新的状态。它在MiniF2F-Test上仅用72次推理预算即可达到97.1%的通过率。在更具有挑战性的基准测试中,它解决了ProverBench和PutnamBench的70.8%和41.5%,显著优于现有开放权重基线。

英文摘要:

We introduce LongCat-Flash-Prover, a flagship 560-billion-parameter open-source Mixture-of- Experts (MoE) model that advances Native Formal Reasoning in Lean4 through agentic tool-integrated reasoning (TIR). We decompose the native formal reasoning task into three independent formal capabilities, i.e., auto-formalization, sketching, and proving. To facilitate these capabilities, we propose a Hybrid-Experts Iteration Framework to expand high-quality task trajectories, including generating a formal statement based on a given informal problem, producing a whole-proof directly from the statement, or a lemma-style sketch. During agentic RL, we present a Hierarchical Importance Sampling Policy Optimization (HisPO) algorithm, which aims to stabilize the MoE model training on such long-horizon tasks. It employs a gradient masking strategy that accounts for the policy staleness and the inherent train-inference engine discrepancies at both sequence and token levels. Additionally, we also incorporate theorem consistency and legality detection mechanisms to eliminate reward hacking issues. Extensive evaluations show that our LongCat-Flash-Prover sets a new state-of-the-art for open-weights models in both auto-formalization and theorem proving. Demonstrating remarkable sample efficiency, it achieves a 97.1% pass rate on MiniF2F-Test using only 72 inference budget per problem. On more challenging benchmarks, it solves 70.8% of ProverBench and 41.5% of PutnamBench with no more than 220 attempts per problem, significantly outperforming existing open-weights baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑