Graph-based Confidence Calibration for Large Language Models
Department of Computer Science Tufts University（塔夫茨大学计算机科学系）；Department of Computer Science Virginia Tech（弗吉尼亚理工大学计算机科学系）；Department of Computer Science Virginia Tech, University of California, Davis（弗吉尼亚理工大学计算机科学系，加州大学戴维斯分校计算机科学系）
Reliable confidence estimation is essential for enhancing the trustworthiness of large language models (LLMs), especially in high-stakes scenarios. Despite its importance, accurately estimating confidence in LLM responses remains a significant challenge. In this work, we propose using an auxiliary learning model to assess response correctness based on the self-consistency of multiple outputs generated by the LLM. Our method builds a consistency graph to represent the agreement among multiple responses and uses a graph neural network (GNN) to estimate the likelihood that each response is correct. Experiments demonstrate that this method has strong calibration performance on various benchmark datasets and generalizes well to out-of-domain cases.
Fact-Level Confidence Calibration and Self-Correction
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Penn State University（宾夕法尼亚州立大学）
Decoupling Reasoning and Confidence: Resurrecting Calibration in Reinforcement Learning from Verifiable Rewards
Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences, Beijing, China（中国科学院软件研究所信息处理实验室）；University of Chinese Academy of Sciences, Beijing, China（中国科学院大学）；Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China（中国科学院大学网络安全学院）；National Computer Network Emergency Response Technical Team/Coordination Center of China, Beijing, China（中国国家计算机网络应急技术配合中心）
Double-Calibration: Towards Reliable LLMs via Calibrating Knowledge and Reasoning Confidence
School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China（中山大学计算机科学与工程学院，广州，中国）；Department of Computing, The Hong Kong Polytechnic University, Hong Kong SAR（香港理工大学计算机系，香港特别行政区）；School of Science and Technology, Hong Kong Metropolitan University, Hong Kong SAR（香港 Metropolitan 大学科技学院，香港特别行政区）
Object-Level Verbalized Confidence Calibration in Vision-Language Models via Semantic Perturbation
University of Science and Technology of China（中国科学技术大学）；SKL of Processors, Institute of Computing Technology, CAS（中国科学院计算技术研究所处理器专项实验室）；National University of Singapore（新加坡国立大学）；University of Chinese Academy of Sciences（中国科学院大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
The Confidence Trap: Calibration Attacks for Graph Neural Networks
Virginia Polytechnic Institute and State University（弗吉尼亚理工学院暨州立大学）；The Pennsylvania State University（宾夕法尼亚州立大学）；VinUniversity；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）
Confidence Calibration for Multimodal LLMs: An Empirical Study through Medical VQA
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；School of Computer Science and Technology, Xidian University（西安电子科技大学计算机科学与技术学院）；Zhihui Medical Technology (Shanghai) Co., Ltd.（智汇医疗科技（上海）有限公司）
Shared Experience, Separate Learning: Companion Confidence Calibration for LLMs
State Key Laboratory of AI Safety（人工智能安全国家重点实验室）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）
N2C2: Nearest Neighbor Enhanced Confidence Calibration for Cross-Lingual In-Context Learning
School of Informatics, University of Edinburgh（爱丁堡大学信息学院）；School of Computer Science and Informatics, Cardiff University（卡迪夫大学计算机科学与信息学院）；College of Intelligence and Computing, Tianjin University（天津大学智能与计算学部）
Claim-Level Confidence Calibration for Reliable Decision Making with Large Language Models
Tsinghua University（清华大学）；Vulcan Research（伏尔坎研究院）；AIFT；Keio Global Research Institute (KGRI)（庆应全球研究所（KGRI））；China Mobile Research Institute（中国移动研究院）；Zhongguancun Laboratory（中关村实验室）
Evaluating and Calibrating LLM Confidence on Questions with Multiple Correct Answers
State Key Laboratory of AI Safety（人工智能安全国家重点实验室）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Tsinghua University（清华大学）；Renmin University of China（中国人民大学）
School of Computer Science, University of Sydney（悉尼大学计算机科学学院）；City University of Hong Kong（香港城市大学）；Shanghai Jiao Tong University（上海交通大学）；University of Oxford, Department of Engineering Science（牛津大学工程科学系）；Department of Engineering Science, University of Oxford（牛津大学工程科学系）
