Accurate confidence estimation is critical for reliable use of large language models (LLMs). Prior work on LLM calibration largely focuses on response-level confidence, which estimates the correctness of a single generated output. However, this formulation is misaligned with many practical settings where the central question is how likely a model is to solve a query overall. We show that this mismatch results from the stochastic nature of modern LLM decoding, under which single-response correctness fails to reflect underlying model capability. To address this issue, we introduce capability calibration, a new evaluation framework for measuring how well query-level confidence aligns with a model's expected accuracy on individual queries. We formally distinguish capability calibration (CC) from response calibration (RC) and show that the two differ both theoretically and empirically. We further show that CC is better suited than RC to applications like pass@k prediction and inference budget allocation. Finally, we evaluate common confidence estimation methods to understand the practical feasibility of CC.
Graph-based Confidence Calibration for Large Language Models
Department of Computer Science Tufts University(塔夫茨大学计算机科学系);Department of Computer Science Virginia Tech(弗吉尼亚理工大学计算机科学系);Department of Computer Science Virginia Tech, University of California, Davis(弗吉尼亚理工大学计算机科学系,加州大学戴维斯分校计算机科学系)
Fact-Level Confidence Calibration and Self-Correction
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Penn State University(宾夕法尼亚州立大学)
Influences on LLM Calibration: A Study of Response Agreement, Loss Functions, and Prompt Styles
Faculty of Computer Science, University of Vienna(维也纳大学计算机学院);UniVie Doctoral School Computer Science(维也纳大学计算机博士学院);Department of Computer Science, Aarhus University(奥胡斯大学计算机系);Faculty of Philological and Cultural Studies, University of Vienna(维也纳大学语文学与文化学院)
Shared Experience, Separate Learning: Companion Confidence Calibration for LLMs
State Key Laboratory of AI Safety(人工智能安全国家重点实验室);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学)
Object-Level Verbalized Confidence Calibration in Vision-Language Models via Semantic Perturbation
University of Science and Technology of China(中国科学技术大学);SKL of Processors, Institute of Computing Technology, CAS(中国科学院计算技术研究所处理器专项实验室);National University of Singapore(新加坡国立大学);University of Chinese Academy of Sciences(中国科学院大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
Probabilistic Calibration Is a Trainable Capability in Language Models
Chandar Research Lab(昌达尔研究实验室);Mila – Quebec AI Institute(魁北克人工智能研究所);LAMA-WeST Lab(LAMA-WeST实验室);Polytechnique Montréal(蒙特利尔理工学院);Université de Montréal(蒙特利尔大学);Independent researcher(独立研究者)
Efficient Uncertainty Estimation with Gaussian Process for Reliable Dialog Response Retrieval
Ping An Technology (Shenzhen) Co., Ltd.(平安科技(深圳)有限公司);University of Science and Technology of China(中国科学技术大学)
深度神经网络在基于检索的对话系统中取得了卓越的性能,但它们被证明是校准不良的。虽然像Monte Carlo Dropout和Entrance这样的基本校准方法可以很好地校准,但这些方法在训练或推理阶段非常耗时。为了应对这些挑战,我们提出了一种用于基于BERT的对话式搜索的高效不确定性校准框架GPF-BERT,该框架在BERT架构之上采用高斯过程层和焦点损失来实现高质量的神经排序器。大量的实验验证了该方法的有效性。与基本标定方法相比,GPF-BERT在三个域内数据集和分布移位任务中实现了最低的经验标定误差(ECE),而在大多数情况下产生了最高的$R_{10}@1$和MAP性能。在时间消耗方面,我们的GPF-BERT具有8 $\times$的加速比。
SGIC: A Self-Guided Iterative Calibration Framework for RAG
NLP 2 CT Lab, Department of Computer and Information Science, University of Macau(自然语言处理2CT实验室,计算机与信息科学系,澳门大学);Institute of Computing and Intelligence, Harbin Institute of Technology, Shenzhen, China(计算与智能学院,哈尔滨工业大学深圳校区,中国)