arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2510.06014cs.AI
收藏

ARISE:用于大型推理模型测试时扩展评估的自适应分辨率感知度量

ARISE: An Adaptive Resolution-Aware Metric for Test-Time Scaling Evaluation in Large Reasoning Models

Fudan University(复旦大学);The University of Hong Kong(香港大学);Nanjing University(南京大学);Shanghai AI Laboratory(上海人工智能实验室);Shanghai Innovation Institute(上海创新研究院)

Zhangyue Yin, Qiushi Sun, Zhiyuan Zeng, Zhiyuan Yu, Qipeng Guo, Xuanjing Huang, Xipeng Qiu

AI总结:

本文提出ARISE度量,通过样本级感知和动态采样机制,可靠评估大型推理模型的测试时扩展能力,并发现Claude Opus扩展特性最优。

摘要:

测试时扩展已成为增强大型推理模型性能的一种变革性范式,能够在推理过程中动态分配计算资源。然而,随着推理模型格局的迅速扩展,一个关键问题仍然存在:我们如何系统地比较和评估不同模型的测试时扩展能力?在本文中,我们引入了ARISE(自适应分辨率感知扩展评估),这是一种专门设计用于评估大型推理模型测试时扩展有效性的新型度量。与现有的评估方法不同,ARISE包含两项关键创新:(1)样本级感知,有效惩罚负向扩展行为,即增加计算量导致性能下降的情况;(2)动态采样机制,减轻精度波动和令牌计数不稳定性对最终评估的影响。我们进行了全面实验,评估了涵盖数学推理、代码生成和智能体任务等多个领域的先进推理模型。我们的结果表明,ARISE提供了对测试时扩展能力的可靠且细粒度的测量,揭示了不同模型在扩展效率上的显著差异。值得注意的是,我们的评估识别出Claude Opus在扩展特性上优于其他当代推理模型。

英文摘要:

Test-time scaling has emerged as a transformative paradigm for enhancing the performance of large reasoning models, enabling dynamic allocation of computational resources during inference. However, as the landscape of reasoning models rapidly expands, a critical question remains: how can we systematically compare and evaluate the test-time scaling capabilities across different models? In this paper, we introduce ARISE (Adaptive Resolution-aware Scaling Evaluation), a novel metric specifically designed to assess the test-time scaling effectiveness of large reasoning models. Unlike existing evaluation approaches, ARISE incorporates two key innovations: (1) sample-level awareness that effectively penalizes negative scaling behaviors where increased computation leads to performance degradation, and (2) a dynamic sampling mechanism that mitigates the impact of accuracy fluctuations and token count instability on the final assessment. We conduct comprehensive experiments evaluating state-of-the-art reasoning models across diverse domains including mathematical reasoning, code generation, and agentic tasks. Our results demonstrate that ARISE provides a reliable and fine-grained measurement of test-time scaling capabilities, revealing significant variations in scaling efficiency across models. Notably, our evaluation identifies Claude Opus as exhibiting superior scaling characteristics compared to other contemporary reasoning models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑