一个用于预测的头与一个用于质疑的头：用于LLM输出中幻觉检测的预训练不确定性量化头
A Head to Predict and a Head to Question: Pre-trained Uncertainty Quantification Heads for Hallucination Detection in LLM Outputs
AI总结：
本文提出预训练的监督式UQ辅助头，利用Transformer架构和LLM注意力图特征检测输出幻觉，在域内外及跨语言场景取得SOTA，并开源模型与代码。
摘要：
大型语言模型（LLM）具有产生幻觉的倾向，即偶尔生成虚假或捏造的信息。这带来了一项重大挑战，因为幻觉往往看起来极具说服力，而用户通常缺乏检测它们的工具。不确定性量化（UQ）提供了一个评估模型输出可靠性的框架，有助于识别潜在幻觉。在本工作中，我们引入了预训练UQ头：用于LLM的监督式辅助模块，与无监督UQ方法相比，它们显著增强了模型捕捉不确定性的能力。其优异性能源于设计中强大的Transformer架构以及从LLM注意力图中提取的信息丰富的特征。实验评估表明，这些头具有高度鲁棒性，并在域内和域外提示的声明级幻觉检测中均取得了最先进性能。此外，这些模块对未经过显式训练的语言表现出很强的泛化能力。我们为流行的LLM系列预训练了一系列UQ头，包括Mistral、Llama和Gemma 2。我们公开发布了代码和预训练头。
英文摘要：
Large Language Models (LLMs) have the tendency to hallucinate, i.e., to sporadically generate false or fabricated information. This presents a major challenge, as hallucinations often appear highly convincing and users generally lack the tools to detect them. Uncertainty quantification (UQ) provides a framework for assessing the reliability of model outputs, aiding in the identification of potential hallucinations. In this work, we introduce pre-trained UQ heads: supervised auxiliary modules for LLMs that substantially enhance their ability to capture uncertainty compared to unsupervised UQ methods. Their strong performance stems from the powerful Transformer architecture in their design and informative features derived from LLM attention maps. Experimental evaluation shows that these heads are highly robust and achieve state-of-the-art performance in claim-level hallucination detection across both in-domain and out-of-domain prompts. Moreover, these modules demonstrate strong generalization to languages they were not explicitly trained on. We pre-train a collection of UQ heads for popular LLM series, including Mistral, Llama, and Gemma 2. We publicly release both the code and the pre-trained heads.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
