arXiv 2511.162752026-06-02cs.CLcs.AI

SeSE: 基于结构信息理论的大语言模型黑盒不确定性量化

SeSE: Black-Box Uncertainty Quantification for Large Language Models Based on Structural Information Theory

School of Cyber Science and Technology Beihang University（北航信息科学与技术学院）；School of Computer Science and Engineering Beihang University（北航计算机科学与工程学院）；Didi Chuxing（滴滴出行）；Laboratory for Big Data and Decision National University of Defense Technology（国防科技大学大数据与决策实验室）；Department of Computer Science University of Illinois Chicago（伊利诺伊大学芝加哥分校计算机科学系）

Xingtao Zhao, Hao Peng, Dingli Su, Xianghua Zeng, Chunyang Liu, Jinzhi Liao, Philip S. Yu

备注：: Accepted by UAI 2026

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出SeSE框架，通过构建语义空间的最优层次抽象并计算结构熵，实现大语言模型的黑盒不确定性量化，理论推广了语义熵并在长文本生成中优于现有方法。

摘要：

可靠的不确定性量化（UQ）对于在安全关键场景中部署大语言模型（LLMs）至关重要，因为它使模型能够在不确定时避免回应，从而避免产生幻觉，即看似合理但事实错误的回应。然而，尽管语义UQ方法取得了先进性能，它们忽略了可能实现更精确不确定性估计的潜在语义结构信息。在本文中，我们提出了语义结构熵（SeSE），一个适用于开源和闭源LLMs的原则性黑盒UQ框架。为了揭示语义空间的内在结构，SeSE通过具有最小结构熵的编码树构建其最优层次抽象。因此，该编码树的结构熵量化了最优压缩后LLM语义空间内的固有不确定性。此外，与主要关注简单短文本生成的现有方法不同，我们将SeSE扩展到为长文本输出提供可解释的、细粒度的不确定性估计。我们从理论上证明SeSE推广了语义熵（LLM中UQ的金标准），并通过24个模型-数据集组合的实验证明其优于强基线的性能。

英文摘要：

Reliable uncertainty quantification (UQ) is essential for deploying large language models (LLMs) in safety-critical scenarios, as it enables them to abstain from responding when uncertain, thereby avoiding hallucinations, i.e., plausible yet factually incorrect responses. However, while semantic UQ methods have achieved advanced performance, they overlook latent semantic structural information that could enable more precise uncertainty estimates. In this paper, we propose \underline{Se}mantic \underline{S}tructural \underline{E}ntropy ({SeSE}), a principled black-box UQ framework applicable to both open- and closed-source LLMs. To reveal the intrinsic structure of the semantic space, SeSE constructs its optimal hierarchical abstraction through an encoding tree with minimal structural entropy. The structural entropy of this encoding tree thus quantifies the inherent uncertainty within LLM semantic space after optimal compression. Additionally, unlike existing methods that primarily focus on simple short-form generation, we extent SeSE to provide interpretable, granular uncertainty estimation for long-form outputs. We theoretically prove that SeSE generalizes semantic entropy, the gold standard for UQ in LLMs, and empirically demonstrate its superior performance over strong baselines across 24 model-dataset combinations.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。