arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.04737cs.AIcs.CLcs.LG
收藏

交互式基准测试

Interactive Benchmarks

InteractiveBench；Princeton University（普林斯顿大学）

Baoqing Yue, Zihan Zhu, Yutong Han, Brian Fan, Qian Sun, Jichen Feng, Hufei Yang, Yifan Zhang, Mengdi Wang

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出交互式基准测试，通过预算化的多轮交互评估模型推理能力，改进传统基准和偏好评估的局限性，揭示模型在交互场景中的改进空间。

摘要：

现有的推理评估范式存在不同局限：固定基准日益饱和且易受污染，而基于偏好的评估依赖主观判断。我们主张智能的核心在于决定获取哪些信息以及如何有效使用它们。我们提出了交互式基准，一种统一的评估范式，通过预算化的多轮交互评估模型的推理能力。我们在两种设置中评估模型：交互证明，其中模型与裁判互动解决逻辑、UI2Html和数学任务，在客观反馈下；以及交互游戏，其中模型战略推理以最大化长期效用。我们的结果表明，交互式基准提供了更稳健的评估，揭示了模型在交互场景中的显著改进空间。

英文摘要：

Existing reasoning evaluation paradigms suffer from different limitations: fixed benchmarks are increasingly saturated and vulnerable to contamination, while preference-based evaluations rely on subjective judgments. We argue that a core aspect of intelligence is the ability to decide what information to acquire and how to use it effectively. We propose Interactive Benchmarks, a unified evaluation paradigm that assesses a model's reasoning ability through budgeted multi-turn interaction. We evaluate models under this framework in two settings: Interactive Proofs, where models interact with a judge to solve Logic, UI2Html, and Mathematics tasks under objective feedback; and Interactive Games, where models reason strategically to maximize long-horizon utilities. Our results show that interactive benchmarks provide a more robust assessment of this dimension of model intelligence, revealing substantial room for improvement in interactive scenarios.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑