深度研究代理中的搜索时污染：衡量公开基准评估中的性能膨胀
Search-Time Contamination in Deep Research Agents: Measuring Performance Inflation in Public Benchmark Evaluation
AI总结：
研究深度研究代理在推理过程中通过网页搜索检索公开基准元数据、问题上下文甚至真实答案导致的搜索时污染（STC），定义三种污染类型并开发检测算法，实验表明STC可导致性能膨胀高达4%。
摘要：
公开基准能够对LLM推理进行公平且可重复的评估，但对于在推理过程中主动搜索网络的深度研究代理而言，这些基准变得脆弱。此类代理可能通过网页搜索检索到公开基准元数据、问题上下文甚至真实答案。这导致了搜索时污染（STC），即外部检索绕过了预期的推理并夸大了测量到的性能。我们系统地研究了深度研究代理评估中的STC。我们定义了三种严重程度递增的污染类型，即基准元数据泄露、问题上下文泄露和显式答案泄露，并开发了检测算法来识别它们并量化其对代理性能的影响。在六个公开基准上评估现代深度研究代理，我们发现STC普遍存在，并可能使性能膨胀高达4%。我们的发现表明，现有的评估可能高估了真实的推理能力。因此，我们提倡污染感知的实践，包括隔离沙箱、透明的搜索轨迹和受控的基准访问。
英文摘要：
Public benchmarks enable fair and reproducible evaluation of LLM reasoning, but they become fragile for deep research agents that actively search the web during inference. Such agents may retrieve public benchmark metadata, question context, or even ground-truth answers via web search. This gives rise to Search-Time Contamination (STC), where external retrieval bypasses intended reasoning and inflates measured performance. We systematically study STC in deep research agent evaluation. We define three contamination types with increasing severity, namely Benchmark Metadata Leakage, Question-Context Leakage, and Explicit Answer Leakage, and develop detection algorithms to identify them and quantify their impact on agent performance. Evaluating modern deep research agents on six public benchmarks, we find that STC is widespread and can inflate performance by up to 4%. Our findings show that existing evaluations may overestimate true reasoning ability. We therefore advocate contamination-aware practices, including isolated sandboxes, transparent search trajectories, and controlled benchmark access.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
