arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.23366cs.AIcs.MA
收藏

GSAR:多智能体大语言模型中基于类型的 grounding 检测与恢复

GSAR: Typed Grounding for Hallucination Detection and Recovery in Multi-Agent LLMs

Agentic AI Oracle Corporation(Agentic AI Oracle公司)

Federico A. Kamelhar

AI总结:

GSAR 提出了一种 grounding 评估与规划框架,通过四类分类、证据权重分配、矛盾惩罚评分和三层决策函数,实现多智能体大语言模型中的 hallucination 检测与恢复。

摘要:

自主多智能体大语言模型系统越来越多地用于调查运营事件并生成结构化诊断报告。其可靠性取决于每个声明是否基于观察到的证据而非模型内部推理。现有 groundedness 评估器(二元分类器、LLM-as-judge 标量、自我纠正循环)将支持证据视为可互换的,并发出单一信号,无法对下游行动提供原则性控制。我们提出了 GSAR,一种 grounding 评估与规划框架,该框架(i)将声明分为四类(grounded、ungrounded、contradicted、complementary),赋予非冗余替代视角以第一类地位;(ii)分配证据类型特定的权重,反映知识强度;(iii)计算一个不对称矛盾惩罚加权 groundedness 分数;(iv)将该分数与三层决策函数(继续、再生、规划)结合,驱动在显式计算预算下的有限迭代外循环。我们正式化了算法,证明了六个结构性性质,并在 FEVER 上使用黄金维基百科证据,通过四个独立训练的 LLM 判官(gpt-5.4、claude-sonnet-4-6、claude-opus-4-7、gemini-2.5-pro)评估了五个设计主张。每项消融实验都在所有判官上以相同方向重现:bootstrap 95% CIs 在 rho=0 效应上排除 0;no-complementary 消融在 Opus 4.7 上有 CI [-96,-68] 的 200;在 n=1000 时,三个独立判官收敛到 DeltaS(rho=0)=+0.058。与 Vectara HHEM-2.1-Open 的头对头比较已包含。据我们所知,GSAR 是第一个发表的 groundedness 框架,结合了证据类型评分与在显式计算预算下的分层恢复。

英文摘要:

Autonomous multi-agent LLM systems are increasingly deployed to investigate operational incidents and produce structured diagnostic reports. Their trustworthiness hinges on whether each claim is grounded in observed evidence rather than model-internal inference. Existing groundedness evaluators (binary classifiers, LLM-as-judge scalars, self-correction loops) treat supporting evidence as interchangeable and emit a single signal that offers no principled control over downstream action. We present GSAR, a grounding-evaluation and replanning framework that (i) partitions claims into a four-way typology (grounded, ungrounded, contradicted, complementary), giving first-class standing to non-redundant alternative perspectives; (ii) assigns evidence-type-specific weights reflecting epistemic strength; (iii) computes an asymmetric contradiction-penalised weighted groundedness score; and (iv) couples that score to a three-tier decision function (proceed, regenerate, replan) driving a bounded-iteration outer loop under an explicit compute budget. We formalise the algorithm, prove six structural properties, and evaluate five design claims on FEVER with gold Wikipedia evidence under four independently-trained LLM judges (gpt-5.4, claude-sonnet-4-6, claude-opus-4-7, gemini-2.5-pro). Every ablation reproduces in the same direction on every judge: bootstrap 95% CIs on the rho=0 effect exclude 0 on all four; the no-complementary ablation under Opus 4.7 has CI [-96,-68] of 200; at n=1000 three independent judges converge to DeltaS(rho=0)=+0.058. A head-to-head against Vectara HHEM-2.1-Open is included. To our knowledge, GSAR is the first published groundedness framework coupling evidence-typed scoring with tiered recovery under an explicit compute budget.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑