ProvenanceGuard: 基于MCP的LLM智能体的源感知事实性验证
ProvenanceGuard: Source-Aware Factuality Verification for MCP-Based LLM Agents
AI总结：
提出ProvenanceGuard，一种源感知验证器，通过追踪MCP工具调用、分解声明并路由到特定源，检测跨源混淆错误，在医疗领域数据集上优于源无关基线。
摘要：
使用工具的LLM智能体越来越多地采用模型上下文协议（MCP）从异构证据源（包括搜索、API、数据库、临床记录和处方工具）中获取答案。标准的事实性指标通常测试答案是否得到汇总证据的支持，但忽略了一种源感知的失败模式：一个声明可能在某个地方得到支持，却被归因于错误的来源。我们称此为跨源混淆。我们引入ProvenanceGuard，一种用于MCP基础答案的源感知验证器。它消耗捕获的MCP轨迹，包含稳定的工具ID、源ID和原始输出；将答案分解为原子声明；将声明路由到特定源的证据；使用NLI和令牌对齐代理检查支持度；比较声明的归属与路由的源；并返回每个声明的判定以及答案级别的允许/阻止决策。被阻止的答案可以通过检索增强的答案修订和重新验证来修复。我们在281个医疗领域的MCP智能体轨迹上进行评估。一个包含266条轨迹的裁定子集产生了2,325个由LLM辅助的声明标签（按轨迹划分）；361个保留标签由人工验证。在40条轨迹的保留子集上，ProvenanceGuard在260个符合源条件的声明上实现了阻止F1分数0.802和源准确率0.858，优于不输出声明到源ID的源无关基线。在一个更困难的多源基准上，它达到了阻止F1分数0.846，而源加关系准确率降至0.229，表明在语义相近的源上精确的源归属仍然困难。修复和重新验证解决了完整轨迹集中的所有被阻止答案，通常通过保守回退。在50个受控的临床混淆探测中，ProvenanceGuard检测到所有注入的归属交换，没有保留错误的归属。这些结果表明，源归属是基于MCP的智能体事实性验证的一个独立维度。
英文摘要：
Tool-using LLM agents increasingly use the Model Context Protocol (MCP) to answer from heterogeneous evidence sources, including search, APIs, databases, clinical records, and formulary tools. Standard factuality metrics usually test whether an answer is supported by pooled evidence, missing a provenance-sensitive failure mode: a claim may be supported somewhere while being attributed to the wrong source. We call this cross-source conflation. We introduce ProvenanceGuard, a source-aware verifier for MCP-grounded answers. It consumes captured MCP traces with stable tool IDs, source IDs, and raw outputs; decomposes answers into atomic claims; routes claims to source-specific evidence; checks support with NLI and a token-alignment proxy; compares stated attribution with the routed source; and returns per-claim verdicts plus an answer-level allow/block decision. Blocked answers can be repaired with retrieval-augmented answer revision and re-verified. We evaluate on 281 medical-domain MCP-agent traces. A 266-trace adjudicated subset yields 2,325 LLM-assisted claim labels split by trace; 361 held-out labels are human-verified. On the 40-trace held-out split, ProvenanceGuard achieves block F1 0.802 and source accuracy 0.858 over 260 source-eligible claims, outperforming source-blind baselines that do not emit claim-to-source IDs. On a harder multi-source benchmark it reaches block F1 0.846, while source-plus-relation accuracy drops to 0.229, showing that exact source ownership remains difficult with semantically close sources. Repair-and-reverify resolves all blocked answers in the full trace set, often via conservative fallback. In 50 controlled clinical conflation probes, ProvenanceGuard detects all injected attribution swaps with no retained wrong attribution. These results show that source attribution is an independent axis for factuality verification in MCP-based agents.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
