工具 receipts,而非零知识证明:面向 AI agent 的实用幻觉检测
Tool Receipts, Not Zero-Knowledge Proofs: Practical Hallucination Detection for AI Agents
AI总结:
NabaOS 通过 epistemic 分类和 receipts 验证,实现交互式 AI agent 幻觉检测的高效高精度验证方法
摘要:
AI agent 通过工具调用执行任务时经常会产生幻觉——伪造工具调用、错误陈述输出数量或把推断当作事实。最近的可验证 AI 推理方法依赖于零知识证明,这些证明提供了密码学保证,但每个查询的证明时间需要几分钟,这使得它们对交互式 agent 来说不切实际。我们提出了 NabaOS,一个受印度认识论(Nyaya Shastra)启发的轻量级验证框架,它将 LLM 响应中的每一个声明按其认识来源(pramana)进行分类:直接工具输出(pratyaksha)、推断(anumana)、外部证词(shabda)、缺席(abhava)或无根据的观点。我们的运行时生成带有 HMAC 签名的工具执行 receipts,LLM 无法伪造这些 receipts,然后通过这些 receipts 跨查声明以实时检测幻觉。我们在 NyayaVerifyBench 上进行了评估,这是一个包含 1800 个 agent 响应场景的新的基准测试,涵盖四种语言,并注入了六种类型的幻觉。NabaOS 检测到 94.2% 的伪造工具引用、87.6% 的数量错误陈述和 91.3% 的虚假缺席声明,每个响应的验证开销低于 15 毫秒。对于深度委托(agent 执行多步骤网络任务),我们的交叉检查协议通过独立重新获取捕获了 78.4% 的 URL 伪造。我们比较了五种方法:zkLLM(密码学证明,180 秒/查询)、TOPLOC(局部敏感哈希)、SPEX(基于采样的执行证明)、张量承诺和自一致性检查。NabaOS 在交互式 agent 的成本-延迟-覆盖率权衡上表现最佳:94.2% 的覆盖率在 <15ms 与 zkLLM 的几乎完美覆盖率在 180,000ms 相比。对于交互式 agent,实用的 receipts 基础验证比密码学证明更具成本效益,认识论分类为用户提供可操作的信任信号,而不是二元判断。
英文摘要:
AI agents that execute tasks via tool calls frequently hallucinate results - fabricating tool executions, misstating output counts, or presenting inferences as facts. Recent approaches to verifiable AI inference rely on zero-knowledge proofs, which provide cryptographic guarantees but impose minutes of proving time per query, making them impractical for interactive agents. We propose NabaOS, a lightweight verification framework inspired by Indian epistemology (Nyaya Shastra), which classifies every claim in an LLM response by its epistemic source (pramana): direct tool output (pratyaksha), inference (anumana), external testimony (shabda), absence (abhava), or ungrounded opinion. Our runtime generates HMAC-signed tool execution receipts that the LLM cannot forge, then cross-references claims against these receipts to detect hallucinations in real time. We evaluate on NyayaVerifyBench, a new benchmark of 1,800 agent response scenarios across four languages with injected hallucinations of six types. NabaOS detects 94.2% of fabricated tool references, 87.6% of count misstatements, and 91.3% of false absence claims, with <15ms verification overhead per response. For deep delegation (agents performing multi-step web tasks), our cross-checking protocol catches 78.4% of URL fabrications via independent re-fetching. We compare against five approaches: zkLLM (cryptographic proofs, 180s/query), TOPLOC (locality-sensitive hashing), SPEX (sampling-based proof of execution), tensor commitments, and self-consistency checking. NabaOS achieves the best cost-latency-coverage trade-off for interactive agents: 94.2% coverage at <15ms versus zkLLM's near-perfect coverage at 180,000ms. For interactive agents, practical receipt-based verification provides better cost-benefit than cryptographic proofs, and epistemic classification gives users actionable trust signals rather than binary judgments.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。