Mind2Web 2：以智能体即裁判评估智能体搜索
Mind2Web 2: Evaluating Agentic Search with Agent-as-a-Judge
AI总结：
本文提出 Mind2Web 2 基准及 Agent-as-a-Judge 框架，通过 130 个长视野任务和树状评分裁判智能体，系统评估智能体搜索系统的答案正确性与来源归因，并揭示 OpenAI Deep Research 已达人类表现 50-70%。
摘要：
智能体搜索（例如 Deep Research 系统）——智能体自主浏览网页、综合信息并返回带有引用的全面答案——代表了用户与网络规模信息交互方式的重大转变。尽管这类系统有望带来更高的效率和认知减负，但智能体搜索日益增长的复杂性和开放性已经超出了现有评估基准和方法所能覆盖的范围，这些基准和方法大多假设较短的搜索视野和静态答案。在本文中，我们介绍了 Mind2Web 2，这是一个包含 130 个真实、高质量且长视野任务的基准，这些任务需要实时网页浏览和广泛的信息综合，构建过程投入了超过 1000 小时的人工劳动。为了解决评估时变且复杂答案的挑战，我们提出了一种新颖的 Agent-as-a-Judge 框架。我们的方法基于树状结构的评分标准设计来构建任务特定的裁判智能体，以自动评估答案正确性和来源归因。我们对十个前沿智能体搜索系统和人类表现进行了全面评估，并进行了详细的错误分析，以期为未来发展提供见解。表现最佳的系统 OpenAI Deep Research 已经能够在花费一半时间的情况下达到人类表现的 50-70%，凸显了其巨大潜力。总体而言，Mind2Web 2 为开发和基准测试下一代智能体搜索系统提供了严格的基础。
英文摘要：
Agentic search such as Deep Research systems-where agents autonomously browse the web, synthesize information, and return comprehensive citation-backed answers-represents a major shift in how users interact with web-scale information. While promising greater efficiency and cognitive offloading, the growing complexity and open-endedness of agentic search have outpaced existing evaluation benchmarks and methodologies, which largely assume short search horizons and static answers. In this paper, we introduce Mind2Web 2, a benchmark of 130 realistic, high-quality, and long-horizon tasks that require real-time web browsing and extensive information synthesis, constructed with over 1000 hours of human labor. To address the challenge of evaluating time-varying and complex answers, we propose a novel Agent-as-a-Judge framework. Our method constructs task-specific judge agents based on a tree-structured rubric design to automatically assess both answer correctness and source attribution. We conduct a comprehensive evaluation of ten frontier agentic search systems and human performance, along with a detailed error analysis to draw insights for future development. The best-performing system, OpenAI Deep Research, can already achieve 50-70% of human performance while spending half the time, highlighting its great potential. Altogether, Mind2Web 2 provides a rigorous foundation for developing and benchmarking the next generation of agentic search systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
