高级数学基准测试：用于高级数学证明生成与验证的基准测试套件

中文摘要 AI 辅助

大语言模型在高中和奥林匹克风格数学上表现出色，但在高级数学方面能力尚不明晰。现有基准测试在范围和评估粒度上不足。为此引入高级数学基准测试套件AdvancedMathBench，其核心证明生成基准ProverBench含296个本科和博士资格考试级别的问题。开发了自动验证管道评估证明，还引入VerifierBench。实验表明该基准测试对前沿模型仍具挑战性，证明生成和验证方面模型都有很大提升空间。

英文摘要

Large language models (LLMs) have achieved remarkable performance on high-school and competition-level mathematics, yet their capabilities on advanced mathematics remain poorly understood. Existing benchmarks, however, fall short in both scope and evaluation granularity: they provide limited disciplinary coverage and often rely on final-answer correctness or coarse judgments, leaving the validity of the reasoning process inadequately assessed. To bridge this gap, we introduce AdvancedMathBench, a benchmark suite designed to evaluate the reasoning capabilities of LLMs on advanced mathematical proofs. Its core generation benchmark, ProverBench, contains 245 problems spanning undergraduate (UG) and doctoral qualifying-exam (QE) levels. To reliably evaluate these proofs, we develop a dedicated automatic verification pipeline that is trained on large-scale expert annotations, produces both correctness verdicts and fine-grained analyses, and exhibits strong agreement with human experts on held-out proof trajectories. We further introduce VerifierBench, consisting of 888 model-generated proof trajectories paired with expert ground truth, to evaluate whether models can correctly judge proof validity and provide sound verification rationales. Experiments show that AdvancedMathBench remains challenging for frontier models. On proof generation, the best-performing model, GPT-5.5-xhigh, achieves only 64.5 and 48.9 on the UG and QE splits, respectively. On proof verification, the best model only attains a Balanced F1 of 65.1. Further analysis reveals a notable mismatch between proof generation and verification capabilities across models.

发表机构

• Shanghai AI Laboratory（上海人工智能实验室）
• MMLab, The Chinese University of Hong Kong（香港中文大学多媒体实验室）
• Shanghai Jiao Tong University（上海交通大学）
• Great Bay University（大湾区大学）

机构由 AI 辅助整理，请以论文原文为准。