arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.02020cs.CLcs.AI
收藏

利用语义图增强不确定性建模以进行幻觉检测

Enhancing Uncertainty Modeling with Semantic Graph for Hallucination Detection

Xiaohongshu Inc.(小红书科技有限公司)

Kedi Chen, Qin Chen, Jie Zhou, Xinqi Tao, Bowen Ding, Jingwen Xie, Mingchen Xie, Peilong Li, Feng Zheng, Liang He

AI总结:

本文提出一种利用语义图增强不确定性建模的幻觉检测方法,通过构建实体与句子关系图进行不确定性传播与校准,在段落级幻觉检测上提升19.78%。

摘要:

大型语言模型(LLMs)容易产生非事实或不忠实的幻觉陈述,这破坏了其在实际场景中的应用。近期研究侧重于基于不确定性的幻觉检测,该方法利用 LLMs 的输出概率进行不确定性计算,且不依赖外部知识或对 LLMs 进行频繁采样。然而,大多数方法仅考虑每个独立 token 的不确定性,而未充分研究 token 与句子之间复杂的语义关系,这限制了对段落中跨越多个 token 和句子的幻觉的检测。在本文中,我们提出了一种利用语义图增强不确定性建模以进行幻觉检测的方法。具体而言,我们首先构建一个能很好地捕捉实体 token 与句子之间关系的语义图。然后,我们结合两个实体之间的关系进行不确定性传播,以增强句子级幻觉检测。鉴于幻觉是由句子间的冲突引起的,我们进一步提出了一种基于图的不确定性校准方法,该方法将句子与其在语义图中邻居的矛盾概率整合到不确定性计算中。在两个数据集上的大量实验表明,我们提出的方法具有巨大优势。特别地,我们在段落级幻觉检测中获得了 19.78% 的显著提升。

英文摘要:

Large Language Models (LLMs) are prone to hallucination with non-factual or unfaithful statements, which undermines the applications in real-world scenarios. Recent researches focus on uncertainty-based hallucination detection, which utilizes the output probability of LLMs for uncertainty calculation and does not rely on external knowledge or frequent sampling from LLMs. Whereas, most approaches merely consider the uncertainty of each independent token, while the intricate semantic relations among tokens and sentences are not well studied, which limits the detection of hallucination that spans over multiple tokens and sentences in the passage. In this paper, we propose a method to enhance uncertainty modeling with semantic graph for hallucination detection. Specifically, we first construct a semantic graph that well captures the relations among entity tokens and sentences. Then, we incorporate the relations between two entities for uncertainty propagation to enhance sentence-level hallucination detection. Given that hallucination occurs due to the conflict between sentences, we further present a graph-based uncertainty calibration method that integrates the contradiction probability of the sentence with its neighbors in the semantic graph for uncertainty calculation. Extensive experiments on two datasets show the great advantages of our proposed approach. In particular, we obtain substantial improvements with 19.78% in passage-level hallucination detection.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑