Riemann-Bench: 面向登月级数学的基准测试
Riemann-Bench: A Benchmark for Moonshot Mathematics
AI总结：
提出Riemann-Bench基准，由专家设计研究级数学问题，评估AI系统超越奥数水平的推理能力，结果显示前沿模型得分低于10%。
摘要：
最近的AI系统在国际数学奥林匹克竞赛中取得了金牌级别的表现，展示了在竞赛式问题解决方面的卓越能力。然而，竞赛数学仅代表了数学推理的一个狭窄部分：问题来自有限的领域，需要最少的先进工具，并且通常奖励洞察力技巧而非深奥的理论知识。我们引入了Riemann-Bench，一个由专家策划的私有基准测试，旨在评估AI系统在研究级数学上的表现，这远远超出了奥林匹克的前沿。问题由常春藤联盟数学教授、研究生和拥有博士学位的IMO金牌得主编写，并且通常需要作者数周才能独立解决。每个问题都经过两位独立领域专家的双盲验证，他们必须从头开始解决问题，并通过程序化验证器得出唯一的封闭形式解。我们将前沿模型评估为不受限制的研究智能体，可以完全访问编码工具、搜索和开放式推理，使用每个问题100次独立运行的无偏统计估计器。我们的结果显示，所有前沿模型目前得分低于10%，揭示了奥林匹克级问题解决与真正研究级数学推理之间的巨大差距。通过保持基准完全私有，我们确保测量的性能反映了真实的数学能力，而不是对训练数据的记忆。
英文摘要：
Recent AI systems have achieved gold-medal-level performance on the International Mathematical Olympiad, demonstrating remarkable proficiency at competition-style problem solving. However, competition mathematics represents only a narrow slice of mathematical reasoning: problems are drawn from limited domains, require minimal advanced machinery, and can often reward insightful tricks over deep theoretical knowledge. We introduce Riemann-Bench, a private benchmark of expert-curated problems designed to evaluate AI systems on research-level mathematics that goes far beyond the olympiad frontier. Problems are authored by Ivy League mathematics professors, graduate students, and PhD-holding IMO medalists, and routinely took their authors weeks to solve independently. Each problem undergoes double-blind verification by two independent domain experts who must solve the problem from scratch, and yields a unique, closed-form solution assessed by programmatic verifiers. We evaluate frontier models as unconstrained research agents, with full access to coding tools, search, and open-ended reasoning, using an unbiased statistical estimator computed over 100 independent runs per problem. Our results reveal that all frontier models currently score below 10%, exposing a substantial gap between olympiad-level problem solving and genuine research-level mathematical reasoning. By keeping the benchmark fully private, we ensure that measured performance reflects authentic mathematical capability rather than memorization of training data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
