arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.08082cs.LGcs.AIeess.SP
收藏

野生环境中代理的频谱护栏:通过注意力拓扑检测工具使用幻觉

Spectral Guardrails for Agents in the Wild: Detecting Tool Use Hallucinations via Attention Topology

Devoteam

Valentin Noël

AI总结:

通过分析注意力拓扑谱,提出一种无需训练数据的防护方法,有效检测代理幻觉,揭示模型错误时注意力噪声特征。

摘要:

在野外部署自主代理需要可靠的防护措施以防止工具使用失败。我们提出了一种基于注意力拓扑谱分析的训练自由护栏,以补充监督方法。在Llama 3.1 8B上,我们的方法通过多特征检测实现了97.7%的召回率,通过81.0%的精确率实现了86.1%的召回率,无需任何标记训练数据。最值得注意的是,我们发现单层谱特征可以作为接近完美的幻觉检测器:Llama L26 Smoothness在单个阈值下实现了98.2%的召回率(213/217个幻觉被捕捉),Mistral L3 Entropy实现了94.7%的召回率。这表明幻觉不仅仅是错误的标记,而是一种热力学状态的变化:当模型出错时,其注意力会变成噪声。通过在匹配领域(N=1000,T=0.3,相同通用领域,幻觉率20-22%)上受控的跨模型评估,我们揭示了“大声骗子”现象:Llama 3.1 8B的失败在频谱上是灾难性的,并且大大更容易检测,而Mistral 7B实现了最佳的区分能力(AUC 0.900)。这些发现确立了谱分析作为代理安全的原理性、高效框架。

英文摘要:

Deploying autonomous agents in the wild requires reliable safeguards against tool use failures. We propose a training free guardrail based on spectral analysis of attention topology that complements supervised approaches. On Llama 3.1 8B, our method achieves 97.7\% recall with multi-feature detection and 86.1\% recall with 81.0\% precision for balanced deployment, without requiring any labeled training data. Most remarkably, we discover that single layer spectral features act as near-perfect hallucination detectors: Llama L26 Smoothness achieves 98.2\% recall (213/217 hallucinations caught) with a single threshold, and Mistral L3 Entropy achieves 94.7\% recall. This suggests hallucination is not merely a wrong token but a thermodynamic state change: the model's attention becomes noise when it errs. Through controlled cross-model evaluation on matched domains ($N=1000$, $T=0.3$, same General domain, hallucination rates 20--22\%), we reveal the ``Loud Liar'' phenomenon: Llama 3.1 8B's failures are spectrally catastrophic and dramatically easier to detect, while Mistral 7B achieves the best discrimination (AUC 0.900). These findings establish spectral analysis as a principled, efficient framework for agent safety.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑