arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.00801cs.AIcs.IR
收藏

合成网络:用于诊断语言代理知识弱点的对抗性定制迷你互联网

The Synthetic Web: Adversarially-Curated Mini-Internets for Diagnosing Epistemic Weaknesses of Language Agents

Microsoft(微软)

Shrey Shah, Levent Ozgur

AI总结:

本研究提出合成网络基准测试,通过对抗性内容测试揭示语言代理在处理冲突信息时的弱点,为高风险领域中的可靠代理开发提供评估框架。

摘要:

语言代理越来越多地作为具备网络功能的系统运作,能够搜索、浏览并综合来自各种来源的信息。然而,这些来源可能包含不可靠或对抗性内容,而代理对对抗性排序的鲁棒性仍缺乏深入理解。现有的基准测试评估功能导航或静态事实性,但无法因果地隔离这种弱点,当前的检索增强生成缓解策略在这样的条件下仍缺乏测试。我们引入了合成网络基准测试,这是一个程序生成的环境,包含成千上万的超链接文章,具有真实标签的可信度和事实性,过程级交互轨迹以及污染过滤器以消除训练数据泄漏。通过在可控的搜索排名中注入一篇高可信度的虚假信息文章,我们测量了对抗性暴露的因果效应,在六个前沿模型中进行了测试。结果揭示了灾难性失败:尽管可以无限访问真实信息源,准确性却崩溃,搜索升级有限且校准严重失当。这些发现暴露了当前前沿模型在处理冲突信息时的根本限制,对高风险领域的部署有即时影响。我们的基准测试使系统分析这些失败模式成为可能,并提供了一个受控的测试平台,用于在对抗性排序下评估缓解策略——这是当前研究中的空白。这项工作为开发鲁棒的搜索和知识谦逊的代理建立了可复现的基准,这些代理能够抵抗在高风险领域中的操纵。

英文摘要:

Language agents increasingly act as web-enabled systems that search, browse, and synthesize information from diverse sources. However, these sources can include unreliable or adversarial content, and the robustness of agents to adversarial ranking - where misleading information appears prominently in search results - remains poorly understood. Existing benchmarks evaluate functional navigation or static factuality but cannot causally isolate this vulnerability, and current mitigation strategies for retrieval-augmented generation remain largely untested under such conditions. We introduce Synthetic Web Benchmark, a procedurally generated environment comprising thousands of hyperlinked articles with ground-truth labels for credibility and factuality, process-level interaction traces, and contamination filtering to eliminate training-data leakage. By injecting a single high-plausibility misinformation article into a controllable search rank, we measure the causal effect of adversarial exposure in six frontier models. The results reveal catastrophic failures: accuracy collapses despite unlimited access to truthful sources, with minimal search escalation and severe miscalibration. These findings expose fundamental limitations in how current frontier models handle conflicting information, with immediate implications for deployment in high-stakes domains. Our benchmark enables systematic analysis of these failure modes and provides a controlled testbed for evaluating mitigation strategies under adversarial ranking - a gap in current research. This work establishes a reproducible baseline for developing search-robust and epistemically humble agents capable of resisting manipulation in high-stakes domains.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑