arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.11570cs.CL
收藏

PRIME: 一种用于数学和工程中可验证推理的过程-结果对齐基准

PRIME: A Process-Outcome Alignment Benchmark for Verifiable Reasoning in Mathematics and Engineering

University of Science and Technology of China(中国科学技术大学);StepFun(阶跃星辰);Tsinghua University(清华大学)

Xiangfeng Wang, Hangyu Guo, Yanlin Lai, Mitt Huang, Liang Zhao, Chengyuan Yao, Yinmin Zhang, Qi Han, Xiaoxiao Ren, Chun Yuan, Tong Xu, Zheng Ge, Xiangyu Zhang, Daxin Jiang

AI总结:

PRIME是一个用于数学和工程中可验证推理的过程-结果对齐基准,通过评估验证器在推导过程中的准确性,提升了强化学习中可验证奖励的训练效果。

摘要:

尽管基于模型的验证器对于扩展具有可验证奖励的强化学习(RLVR)至关重要,但当前以结果为中心的验证范式主要关注最终结果与地面真实值之间的一致性,往往忽视了推导过程中的潜在错误。这导致了对由错误推导产生的正确答案给予正奖励。为弥合这一差距,我们引入了PRIME,这是一个用于评估数学和工程中过程-结果对齐验证的基准。PRIME从全面的大学级STEM问题集合中精心挑选出2530个高难度样本,通过基于一致性的过滤流程进行筛选。通过广泛评估,我们发现当前验证器经常无法检测到推导中的错误。此外,我们提出了一种过程感知的RLVR训练范式,利用通过PRIME选择的验证器。这种方法在AIME24、AIME25和Beyond-AIME上分别实现了8.29%、9.12%和7.31%的绝对性能提升,对于Qwen3-14B-Base模型而言。最后,我们展示了验证器在PRIME上的准确性与RLVR训练有效性之间存在很强的线性相关性(R²>0.92),验证了PRIME作为验证器选择可靠预测器的作用。

英文摘要:

While model-based verifiers are essential for scaling Reinforcement Learning with Verifiable Rewards (RLVR), current outcome-centric verification paradigms primarily focus on the consistency between the final result and the ground truth, often neglecting potential errors in the derivation process. This leads to assigning positive rewards to correct answers produced from incorrect derivations. To bridge this gap, we introduce PRIME, a benchmark for evaluating verifiers on Process-Outcome Alignment verification in Mathematics and Engineering. Curated from a comprehensive collection of college-level STEM problems, PRIME comprises 2,530 high-difficulty samples through a consistency-based filtering pipeline. Through extensive evaluation, we find that current verifiers frequently fail to detect derivation flaws. Furthermore, we propose a process-aware RLVR training paradigm utilizing verifiers selected via PRIME. This approach substantially outperforms the outcome-only verification baseline, achieving absolute performance gains of 8.29%, 9.12%, and 7.31% on AIME24, AIME25, and Beyond-AIME, respectively, for the Qwen3-14B-Base model. Finally, we demonstrate a strong linear correlation ($R^2 > 0.92$) between verifier accuracy on PRIME and RLVR training effectiveness, validating PRIME as a reliable predictor for verifier selection.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑