iAgentBench: 信息检索代理在高流量话题上的理解能力基准测试
iAgentBench: Benchmarking Sensemaking Capabilities of Information-Seeking Agents on High-Traffic Topics
AI总结：
iAgentBench通过整合多源证据来评估信息检索代理在高流量话题中的理解能力，强调证据使用而非单纯检索的重要性。
摘要：
随着能够搜索的生成问答系统出现，用户越来越倾向于使用工具来浏览、汇总和整合多个来源的信息。然而，许多广泛使用的问答基准测试仍然只能通过检索单个相关段落来回答，这使它们难以衡量跨源理解能力，如整合证据、跟踪因果联系以及在话题的不同方面中解决依赖关系。我们提出了iAgentBench，一个动态的开放域问答基准测试，旨在针对这些高级信息需求，同时保持问题自然且基于现实的信息检索行为。iAgentBench从真实世界的关注信号中提取种子话题，并利用常见的用户意图模式构建用户样式的疑问，其答案需要整合多个来源的证据，而不仅仅是提取单个片段。每个实例都附带可追溯的证据和可审计的中间成果，以支持污染检查并允许对检索与合成中的故障进行细粒度诊断。在多个大型语言模型上的实验表明，检索提高了准确性，但仅依赖检索无法可靠地解决这些问题，这凸显了评估证据使用而非仅仅证据访问的必要性。
英文摘要：
With the emergence of search-enabled generative QA systems, users are increasingly turning to tools that browse, aggregate, and reconcile evidence across multiple sources on their behalf. Yet many widely used QA benchmarks remain answerable by retrieving a single relevant passage, making them poorly suited for measuring cross-source sensemaking, such as integrating evidence, tracking causal links, and resolving dependencies across facets of a topic. We present iAgentBench, a dynamic ODQA benchmark that targets these higher-level information needs while keeping questions natural and grounded in realistic information-seeking behavior. iAgentBench draws seed topics from real-world attention signals and uses common user intent patterns to construct user-like questions whose answers require combining evidence from multiple sources, not just extracting a single snippet. Each instance is released with traceable evidence and auditable intermediate artifacts that support contamination checks and enable fine-grained diagnosis of failures in retrieval versus synthesis. Experiments across multiple LLMs show that retrieval improves accuracy, but retrieval alone does not reliably resolve these questions, underscoring the need to evaluate evidence use, not just evidence access.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
