arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.15927cs.CLcs.AIcs.LG
收藏

语义熵探针:大语言模型中稳健且低成本的幻觉检测方法

Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMs

University of Oxford(牛津大学);OATML

Jannik Kossen, Jiatong Han, Muhammed Razzak, Lisa Schut, Shreshth Malik, Yarin Gal

AI总结:

针对大语言模型幻觉检测中语义熵计算成本过高的问题,提出可从单次生成隐藏状态直接近似语义熵的语义熵探针,在保持检测性能的同时大幅降低开销,且泛化性更优。

摘要:

我们提出了语义熵探针(Semantic Entropy Probes, SEPs),这是一种用于大语言模型(Large Language Models, LLMs)不确定性量化的低成本可靠方法。幻觉指的是听起来合理但事实错误且随意的模型生成内容,是LLM实际应用面临的主要挑战。Farquhar等人(2024)的近期研究提出了语义熵(semantic entropy, SE),可通过估计一组模型生成内容在语义空间中的不确定性来检测幻觉,但SE计算带来的5到10倍计算成本增长阻碍了其实际应用。为解决这一问题,我们提出SEPs,可从单次生成的隐藏状态中直接近似SE。SEPs训练简单,测试时无需采样多个模型生成结果,将语义不确定性量化的开销降至几乎为零。我们证明,SEPs在幻觉检测上保持了高性能,且相较于直接预测模型准确率的既往探针方法,对分布外数据的泛化性更好。我们在不同模型和任务上的结果表明,模型隐藏状态能够捕获SE,消融实验进一步揭示了这一特性对应的token位置和模型层。

英文摘要:

We propose semantic entropy probes (SEPs), a cheap and reliable method for uncertainty quantification in Large Language Models (LLMs). Hallucinations, which are plausible-sounding but factually incorrect and arbitrary model generations, present a major challenge to the practical adoption of LLMs. Recent work by Farquhar et al. (2024) proposes semantic entropy (SE), which can detect hallucinations by estimating uncertainty in the space semantic meaning for a set of model generations. However, the 5-to-10-fold increase in computation cost associated with SE computation hinders practical adoption. To address this, we propose SEPs, which directly approximate SE from the hidden states of a single generation. SEPs are simple to train and do not require sampling multiple model generations at test time, reducing the overhead of semantic uncertainty quantification to almost zero. We show that SEPs retain high performance for hallucination detection and generalize better to out-of-distribution data than previous probing methods that directly predict model accuracy. Our results across models and tasks suggest that model hidden states capture SE, and our ablation studies give further insights into the token positions and model layers for which this is the case.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑