arXiv 2509.014552026-06-15cs.CL

大型语言模型中的可信不确定性：置信度校准与风险控制拒绝的统一框架

Trusted Uncertainty in Large Language Models: A Unified Framework for Confidence Calibration and Risk-Controlled Refusal

University of Liechtenstein（列支敦士登大学）；University of the Republic of San Marino（圣马尔科共和国大学）；University of Mauritius（毛里求斯大学）；International University of Monaco（摩纳哥国际大学）；University of Andorra（安道尔大学）；University of Gibraltar（直布罗陀大学）；University of the Faroe Islands（法罗群岛大学）；Ilisimatusarfik (University of Greenland)（格陵兰岛研究所（格陵兰大学））；University of Prizren（普里兹伦大学）

Markus Oehri, Giulia Conti, Kaviraj Pather, Alexandre Rossi, Laia Serra, Adrian Parody, Rogvi Johannesen, Aviaja Petersen, Arben Krasniqi

备注：: arXiv admin note: This paper has been withdrawn by arXiv due to unverifiable authorship and affiliation

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出UniCR框架，融合多种不确定性证据为校准的正确概率，并通过原则性拒绝满足用户指定的错误预算，无需微调基础模型，在短问答、代码生成和检索增强长问答中提升校准指标并降低风险-覆盖曲线下面积。

摘要：

部署的语言模型必须决定回答什么以及何时不回答。我们提出UniCR，一个统一框架，将异构的不确定性证据（包括序列似然、自一致性离散度、检索兼容性以及工具或验证器反馈）转化为校准的正确概率，然后通过原则性拒绝强制执行用户指定的错误预算。UniCR学习一个轻量级的校准头，采用温度缩放和适当评分，通过黑盒特征支持仅API模型，并使用共形风险控制提供无分布保证。对于长文本生成，我们通过监督从检索证据中得出的原子事实性分数，将置信度与语义保真度对齐，在保持覆盖的同时减少自信幻觉。在短问答、带执行测试的代码生成和检索增强长问答上的实验显示，与熵或logit阈值、后处理校准器和端到端选择性基线相比，校准指标持续改善，风险-覆盖曲线下面积更低，固定风险下覆盖率更高。分析表明，证据矛盾、语义离散度和工具不一致是弃权的主要驱动因素，产生信息丰富的面向用户的拒绝消息。结果是一种可移植的证据融合到校准概率再到风险控制决策的配方，无需微调基础模型即可提高可信度，并在分布偏移下保持有效。

英文摘要：

Deployed language models must decide not only what to answer but also when not to answer. We present UniCR, a unified framework that turns heterogeneous uncertainty evidence including sequence likelihoods, self-consistency dispersion, retrieval compatibility, and tool or verifier feedback into a calibrated probability of correctness and then enforces a user-specified error budget via principled refusal. UniCR learns a lightweight calibration head with temperature scaling and proper scoring, supports API-only models through black-box features, and offers distribution-free guarantees using conformal risk control. For long-form generation, we align confidence with semantic fidelity by supervising on atomic factuality scores derived from retrieved evidence, reducing confident hallucinations while preserving coverage. Experiments on short-form QA, code generation with execution tests, and retrieval-augmented long-form QA show consistent improvements in calibration metrics, lower area under the risk-coverage curve, and higher coverage at fixed risk compared to entropy or logit thresholds, post-hoc calibrators, and end-to-end selective baselines. Analyses reveal that evidence contradiction, semantic dispersion, and tool inconsistency are the dominant drivers of abstention, yielding informative user-facing refusal messages. The result is a portable recipe of evidence fusion to calibrated probability to risk-controlled decision that improves trustworthiness without fine-tuning the base model and remains valid under distribution shift.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。