HIDE 与 Seek：通过解耦表示检测语言模型中的幻觉
HIDE and Seek: Detecting Hallucinations in Language Models via Decoupled Representations
AI总结：
本文提出单次通过、无需训练的HIDE方法，通过量化语言模型内部表示与输出的统计解耦来检测幻觉，在多个数据集上优于现有单次方法，并接近多次方法，显著降低计算成本。
摘要：
当代语言模型（LMs）虽然流畅度惊人，但常常生成与事实不符或对输入上下文不忠实的内容——这一关键问题通常被称为“幻觉”。语言模型生成幻觉内容的倾向削弱了其可靠性，尤其是因为这些捏造内容往往极具说服力，因此难以检测。虽然现有多种方法尝试检测幻觉，但大多数依赖于对每个输入分析多次生成，导致计算成本和延迟增加。为解决此问题，我们提出了一种单次通过、无需训练的有效幻觉检测方法，即通过解耦表示进行检测（HIDE）。我们的方法利用了以下假设：幻觉源于语言模型对输入上下文的内部表示与其生成输出之间的统计解耦。我们使用希尔伯特-施密特独立性准则（HSIC）量化这种解耦，该准则应用于生成输出序列时提取的隐藏状态表示。我们在四个多样化的问答数据集上进行了广泛实验，评估了六种不同规模和属性的开源语言模型上的忠实性和事实性幻觉。我们的结果表明，HIDE在几乎所有设置中均优于其他单次通过方法，在各种模型和数据集上，相对于最佳单次通过策略，AUC-ROC平均相对提升约29%。此外，HIDE与多次通过的最先进方法相比表现出竞争性且通常更优的性能，在AUC-ROC上获得约3%的平均相对提升，同时消耗约51%更少的计算时间。我们的发现强调了利用语言模型中内部表示解耦进行高效实用幻觉检测的有效性。
英文摘要：
Contemporary Language Models (LMs), while impressively fluent, often generate content that is factually incorrect or unfaithful to the input context - a critical issue commonly referred to as 'hallucination'. This tendency of LMs to generate hallucinated content undermines their reliability, especially because these fabrications are often highly convincing and therefore difficult to detect. While several existing methods attempt to detect hallucinations, most rely on analyzing multiple generations per input, leading to increased computational cost and latency. To address this, we propose a single-pass, training-free approach for effective Hallucination detectIon via Decoupled rEpresentations (HIDE). Our approach leverages the hypothesis that hallucinations result from a statistical decoupling between an LM's internal representations of input context and its generated output. We quantify this decoupling using the Hilbert-Schmidt Independence Criterion (HSIC) applied to hidden-state representations extracted while generating the output sequence. We conduct extensive experiments on four diverse question answering datasets, evaluating both faithfulness and factuality hallucinations across six open-source LMs of varying scales and properties. Our results demonstrate that HIDE outperforms other single-pass methods in almost all settings, achieving an average relative improvement of ~29% in AUC-ROC over the best-performing single-pass strategy across various models and datasets. Additionally, HIDE shows competitive and often superior performance with multi-pass state-of-the-art methods, obtaining an average relative improvement of ~3% in AUC-ROC while consuming ~51% less computation time. Our findings highlight the effectiveness of exploiting internal representation decoupling in LMs for efficient and practical hallucination detection.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
