具有可证明对齐保证的大语言模型中的不确定性感知弃权
Uncertainty-Aware Abstention in Large Language Models with Provable Alignment Guarantees
AI总结:
研究大语言模型在问答系统中无可靠置信估计的问题,提出基于置信区间的校准框架CIC,将任意不确定性分数转化为风险可控的选择性回答规则,保证所选阈值控制错误率,兼具风险控制和回答效率。
摘要:
大语言模型(LLMs)越来越多地部署在问答(QA)系统中,但可能产生幻觉或错误对齐的回答且无可靠置信估计。不确定性量化(UQ)为选择性回答提供基础。现有LLMs不确定性分数常是启发式的。我们提出CIC,一种基于置信区间的校准框架,将任意不确定性分数转换为风险控制的选择性回答规则……
英文摘要:
Large language models (LLMs) are increasingly deployed in question answering (QA) systems, yet they may generate hallucinated or misaligned responses without reliable confidence estimates. Uncertainty quantification (UQ) offers a natural basis for selective answering, where a system answers only when its prediction is deemed reliable and abstains otherwise. However, existing uncertainty scores for LLMs are often heuristic: a threshold chosen on such scores does not, by itself, provide statistical guarantees on the error rate among accepted answers. We propose CIC, a confidence-interval-based calibration framework that converts arbitrary uncertainty scores into risk-controlled selective answering rules. Given a held-out calibration set, CIC evaluates each generated response using an application-specific alignment criterion and associates it with an uncertainty score and a binary error label. For each candidate uncertainty threshold, CIC estimates the acceptance-conditioned error rate and constructs a high-probability upper confidence bound using either Hoeffding-style or Clopper-Pearson confidence intervals. It then selects the largest threshold whose upper bound is below a user-specified risk level $α$, thereby maximizing the answering rate subject to a finite-sample reliability constraint. Under exchangeability, CIC guarantees with probability at least $1-δ$ that the selected threshold, if non-null, controls the error rate among accepted answers at level $α$. We evaluate CIC on both closed-ended and open-ended QA benchmarks across seven LLMs and multiple uncertainty estimators. Experimental results show that CIC consistently achieves valid risk control while retaining strong answering efficiency, providing a practical and statistically grounded mechanism for deploying LLMs in reliability-sensitive QA workflows.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。