INSIDE：大语言模型内部状态保留幻觉检测能力
INSIDE: LLMs' Internal States Retain the Power of Hallucination Detection
AI总结：
本文提出INSIDE方法，利用大语言模型内部状态中的稠密语义信息进行幻觉检测，并设计EigenScore指标和测试时特征裁剪策略，在多个模型和问答基准上验证了其有效性。
摘要：
知识幻觉已引发对部署的大语言模型安全性与可靠性的广泛担忧。此前在幻觉检测方面的工作主要采用logit级别的不确定性估计或语言级别的自洽性评估，而在token解码过程中语义信息不可避免地会丢失。因此，我们提出探索大语言模型内部状态中保留的稠密语义信息用于幻觉检测（INSIDE）。具体而言，我们提出一种简单而有效的EigenScore指标，以更好地评估响应的自洽性，该指标利用响应协方差矩阵的特征值来衡量稠密嵌入空间中的语义一致性/多样性。此外，从自洽幻觉检测的角度出发，我们探索了一种测试时特征裁剪方法，用于截断内部状态中的极端激活，从而减少过度自信的生成，并可能有助于检测过度自信的幻觉。我们在多个流行的大语言模型和问答（QA）基准上进行了大量实验和消融研究，验证了所提方法的有效性。
英文摘要：
Knowledge hallucination have raised widespread concerns for the security and reliability of deployed LLMs. Previous efforts in detecting hallucinations have been employed at logit-level uncertainty estimation or language-level self-consistency evaluation, where the semantic information is inevitably lost during the token-decoding procedure. Thus, we propose to explore the dense semantic information retained within LLMs' \textbf{IN}ternal \textbf{S}tates for halluc\textbf{I}nation \textbf{DE}tection (\textbf{INSIDE}). In particular, a simple yet effective \textbf{EigenScore} metric is proposed to better evaluate responses' self-consistency, which exploits the eigenvalues of responses' covariance matrix to measure the semantic consistency/diversity in the dense embedding space. Furthermore, from the perspective of self-consistent hallucination detection, a test time feature clipping approach is explored to truncate extreme activations in the internal states, which reduces overconfident generations and potentially benefits the detection of overconfident hallucinations. Extensive experiments and ablation studies are performed on several popular LLMs and question-answering (QA) benchmarks, showing the effectiveness of our proposal.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
