部分证据基准：在代理系统中受授权限制的证据基准测试
Partial Evidence Bench: Benchmarking Authorization-Limited Evidence in Agentic Systems
AI总结：
本文提出Partial Evidence Bench，用于评估代理系统在授权限制下的证据处理能力，通过四个维度评估系统表现，展示不同行为模式对完成度的影响。
摘要：
企业代理越来越多地在受限制的检索系统、委托工作流和政策约束的证据环境中运行。在这些设置中，可以通过正确执行访问控制来确保系统仍然产生一个看似完整的答案，即使材料证据位于调用者的授权边界之外。本文引入了Partial Evidence Bench，一个确定性的基准，用于衡量这种失败模式。该基准包含三个场景家族——尽职调查、合规审计和安全事件响应，共72个任务，ACL划分的语料库、 oracle完整答案、 oracle授权视图答案、 oracle完整性判断以及结构化缺口报告oracle。它评估系统在四个维度：答案正确性、完整性意识、缺口报告质量以及不安全的完整性行为。检查进的基线显示，在所有发布的家族中，静默过滤在所有情况下都是灾难性不安全的，而显式失败并报告行为可以消除不安全的完整性而不导致任务陷入琐碎的回避。初步的现实模型运行显示，系统在是否过度声称完整性、保守地低估或以企业可使用的形式报告不完整性方面存在模型依赖性和场景敏感性差异。该基准的更广泛贡献是使治理关键的代理失败可测量，而无需人类法官或易受污染的静态语料库。
英文摘要：
Enterprise agents increasingly operate inside scoped retrieval systems, delegated workflows, and policy-constrained evidence environments. In these settings, access control can be enforced correctly while the system still produces an answer that appears complete even though material evidence lies outside the caller's authorization boundary. This paper introduces Partial Evidence Bench, a deterministic benchmark for measuring that failure mode. The benchmark ships three scenario families -- due diligence, compliance audit, and security incident response -- with 72 tasks total, ACL-partitioned corpora, oracle complete answers, oracle authorized-view answers, oracle completeness judgments, and structured gap-report oracles. It evaluates systems along four surfaces: answer correctness, completeness awareness, gap-report quality, and unsafe completeness behavior. Checked-in baselines show that silent filtering is catastrophically unsafe across all shipped families, while explicit fail-and-report behavior eliminates unsafe completeness without collapsing the task into trivial abstention. Preliminary real-model runs show model-dependent and scenario-sensitive differences in whether systems overclaim completeness, conservatively underclaim, or report incompleteness in an enterprise-usable form. The benchmark's broader contribution is to make a governance-critical agent failure measurable without human judges or contamination-prone static corpora.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
