EvoBrowseComp: 基于演化知识的搜索智能体基准测试
EvoBrowseComp: Benchmarking Search Agents on Evolving Knowledge
AI总结：
提出EvoBrowseComp，一个通过实时网络遍历自动生成400道英文和400道中文无污染复杂问题的演化基准，用于评估搜索智能体在动态知识环境中的真实浏览能力。
摘要：
搜索智能体——即增强搜索工具的大型语言模型——加剧了对未来验证基准的需求。现有的基准如BrowseComp依赖静态知识，容易受到测试集污染和参数记忆的影响。因此，模型可以通过事实回忆而非真正检索获得高分，通过推理捷径掩盖真实的浏览能力。在本文中，我们介绍EvoBrowseComp，一个包含400道英文和400道中文无污染复杂问题的演化基准，通过实时网络遍历合成。为了收集这些问题，我们设计了一个三智能体协作框架：（1）QA合成智能体，从实时网络中检索新鲜知识以合成问答对；（2）信息过滤智能体，根据可信度和流行度过滤检索到的知识，以阻断参数捷径；（3）高级指导智能体，将问题形式化为推理图，以减少合成问答对中的逻辑冗余和捷径。由于该框架支持全自动合成，EvoBrowseComp可以定期更新以防止数据污染并保持时间新鲜度。大量实验证实了其高难度，需要广泛的横向搜索。它为自动更新、高难度的基准测试建立了一个可扩展的范式，与不断发展的世界知识和不断进步的智能体能力保持同步。
英文摘要：
Search Agents -- large language models augmented with search tools -- have intensified the need for future-proof evaluation benchmarks. Existing benchmarks such as BrowseComp rely on static knowledge, making them vulnerable to test-set contamination and parametric memorization. Consequently, models can achieve high scores through fact recall rather than genuine retrieval, obscuring true browsing competence via reasoning shortcuts. In this paper, we introduce EvoBrowseComp, an evolving benchmark of 400 English and 400 Chinese contamination-free complex questions synthesized via live-web traversal. To collect these questions, we design a three-agent collaborative framework: (1) a QA synthesis agent that retrieves fresh knowledge from the live web to synthesize QA pairs; (2) an information filtering agent that filters retrieved knowledge in terms of credibility and popularity to block parametric shortcuts; and (3) a high-level guidance agent that formalizes questions into reasoning graphs to reduce logical redundancy and shortcuts in synthesized QA pairs. Because the framework supports fully automated synthesis, EvoBrowseComp can be regularly updated to prevent data contamination and maintain temporal freshness. Extensive experiments confirm its great difficulty, requiring broad horizontal search. It establishes a scalable paradigm for auto-updatable, high-difficulty benchmarking that keeps pace with both evolving world knowledge and advancing agent capabilities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
