arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.01400cs.CL
收藏

EternalMath: 一个与人类发现同步演化的前沿数学基准

EternalMath: A Living Benchmark of Frontier Mathematics that Evolves with Human Discovery

School of Mathematics, Renmin University of China(中国人民大学数学学院);Tencent(腾讯)

Jicheng Ma, Guohua Wang, Xinhua Feng, Yiming Liu, Zhichao Hu, Yuhong Liu

AI总结:

本文提出一个自动化数学推理评估框架,通过将最新数学文献转化为可执行任务,构建可扩展的EternalMath基准,揭示LLM在前沿数学上的性能差距。

摘要:

当前大型语言模型的数学推理评估主要依赖静态基准,导致研究级数学覆盖有限且性能迅速饱和。本文提出一个完全自动化的、基于定理的评估流程,直接将最新同行评审数学文献转化为可执行且可验证的推理任务。该流程识别构造性或定量结果,将其实例化为参数化问题模板,并通过执行验证生成确定性解决方案,从而实现可扩展、可重复和持续更新的评估,无需依赖大规模专家编写。通过设计,该方法支持时间扩展性、内在正确性检查和数学子领域的定制化。应用该流程得到EternalMath,一个从当代研究论文中衍生的演进评估套件。对最新LLM的实验揭示了显著的性能差距,表明前沿数学推理仍远未饱和,凸显了需要与人类数学发现同步进化的评估方法的重要性。

英文摘要:

Current evaluations of mathematical reasoning in large language models (LLMs) are dominated by static benchmarks, either derived from competition-style problems or curated through costly expert effort, resulting in limited coverage of research-level mathematics and rapid performance saturation. We propose a fully automated, theorem-grounded pipeline for evaluating frontier mathematical reasoning, which directly transforms recent peer-reviewed mathematical literature into executable and verifiable reasoning tasks. The pipeline identifies constructive or quantitative results, instantiates them into parameterized problem templates, and generates deterministic solutions through execution-based verification, enabling scalable, reproducible, and continuously updatable evaluation without reliance on large-scale expert authoring. By design, this approach supports temporal extensibility, intrinsic correctness checking, and domain-specific customization across mathematical subfields. Applying this pipeline yields \textbf{EternalMath}, an evolving evaluation suite derived from contemporary research papers. Experiments with state-of-the-art LLMs reveal substantial performance gaps, indicating that mathematical reasoning at the research frontier remains far from saturated and underscoring the need for evaluation methodologies that evolve in step with human mathematical discovery.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑