arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.07528cs.CLcs.AIcs.LG
收藏

BEACON: 面向大语言模型跨模型幻觉检测的行为熵聚合

BEACON: Behavioral Entropy Aggregation for Cross-Model Hallucination Detection in Large Language Models

LLM Lens

Naveen Bera, Pulijala Sai Nikhila, Kondaguduru Abhiram, Shaik Gayaz Ali, Shoaib Sadiq Salehmohamed, Shaik Mohammed Omar, Jinal Prashant Thakkar, Hansika Aredla, Shalmali Ayachit

AI总结:

提出BEACON框架,通过多维度行为特征(语义熵、嵌入几何、思维链一致性、释义稳定性)的黑盒检测方法,在7个基准上达到0.8123 AUROC,优于现有方法。

摘要:

大语言模型中的幻觉,即生成事实上不正确或未经支持的内容,仍然是可靠部署的关键障碍。我们提出了BEACON(面向跨模型幻觉检测的行为熵聚合),一个黑盒幻觉检测框架,仅基于模型输出运行,无需访问内部表示或外部知识库。BEACON从结构化的多遍生成中提取31维特征向量,整合了基于NLI的语义熵、嵌入几何、思维链一致性和释义稳定性信号。在七个基准的7,617个标记样本上训练的梯度提升分类器达到了0.8123 ± 0.0102的AUROC(95%置信区间:0.7632-0.8251),优于独立的语义熵(+0.2298)和SelfCheckGPT风格的一致性基线(+0.2457)。特征重要性分析表明,幻觉本质上是多维的,需要组合的不确定性信号。一个高效的5次调用变体达到了0.7795的AUROC,使得在黑盒LLM API上的实际部署成为可能。

英文摘要:

Hallucination in large language models (LLMs), defined as the generation of factually incorrect or unsupported content, remains a critical barrier to reliable deployment. We present BEACON (Behavioral Entropy Aggregation for Cross-model hallucination detectiON), a black-box hallucination detection framework that operates purely on model outputs without requiring access to internal representations or external knowledge bases. BEACON extracts a 31-dimensional feature vector from structured multi-pass generation, integrating NLI-based semantic entropy, embedding geometry, chain-of-thought consistency, and paraphrase stability signals. A gradient-boosted classifier trained on 7,617 labeled examples across seven benchmarks achieves 0.8123 +/- 0.0102 AUROC (95% CI: 0.7632-0.8251), outperforming standalone semantic entropy (+0.2298) and SelfCheckGPT-style consistency baselines (+0.2457). Feature importance analysis shows that hallucination is inherently multi-dimensional, requiring combined uncertainty signals. An efficient 5-call variant achieves 0.7795 AUROC, enabling practical deployment across black-box LLM APIs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑