语义体积：量化和检测大型语言模型（LLMs）中的外部与内部不确定性
Semantic Volume: Quantifying and Detecting both External and Internal Uncertainty in LLMs
AI总结：
针对LLMs幻觉源于外部（查询模糊）和内部（知识缺失/冲突）不确定性的问题，提出语义体积度量方法，通过扰动嵌入计算Gram矩阵行列式检测不确定性，实验优于基线且有理论支撑，提升模型可靠性。
摘要：
大型语言模型（LLMs）通过编码海量事实知识在各类任务中展现出卓越性能，但仍易产生幻觉，生成错误或误导性信息，且常伴随高不确定性。现有幻觉检测方法主要聚焦于量化内部不确定性，即模型内部知识缺失或冲突引发的不确定性；然而幻觉也可能源于外部不确定性，即模糊的用户查询导致多种可能的解释。本研究提出“语义体积”这一新颖数学度量方法，用于量化LLMs中的外部与内部不确定性。我们的方法通过扰动查询和响应，将其嵌入语义空间，计算嵌入向量的Gram矩阵行列式，以捕捉其离散程度作为不确定性度量。该框架提供了一种可泛化的无监督不确定性检测方法，无需访问LLMs的内部结构。我们针对外部和内部不确定性检测开展了大量实验，结果表明语义体积方法在两项任务中均持续优于现有基线。此外，我们提供了将该度量与微分熵关联的理论见解，统一并扩展了先前基于采样的不确定性度量（如语义熵）。语义体积被证明是一种鲁棒且可解释的方法，通过系统检测用户查询和模型响应中的不确定性，提升LLMs的可靠性。
英文摘要：
Large language models (LLMs) have demonstrated remarkable performance across diverse tasks by encoding vast amounts of factual knowledge. However, they are still prone to hallucinations, generating incorrect or misleading information, often accompanied by high uncertainty. Existing methods for hallucination detection primarily focus on quantifying internal uncertainty, which arises from missing or conflicting knowledge within the model. However, hallucinations can also stem from external uncertainty, where ambiguous user queries lead to multiple possible interpretations. In this work, we introduce Semantic Volume, a novel mathematical measure for quantifying both external and internal uncertainty in LLMs. Our approach perturbs queries and responses, embeds them in a semantic space, and computes the Gram matrix determinant of the embedding vectors, capturing their dispersion as a measure of uncertainty. Our framework provides a generalizable and unsupervised uncertainty detection method without requiring internal access to LLMs. We conduct extensive experiments on both external and internal uncertainty detections, demonstrating that our Semantic Volume method consistently outperforms existing baselines in both tasks. Additionally, we provide theoretical insights linking our measure to differential entropy, unifying and extending previous sampling-based uncertainty measures such as the semantic entropy. Semantic Volume is shown to be a robust and interpretable approach to improving the reliability of LLMs by systematically detecting uncertainty in both user queries and model responses.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
