To enhance Large Language Models' (LLMs) reliability, calibration is essential -- the model's assessed confidence scores should align with the actual likelihood of its responses being correct. However, current confidence elicitation methods and calibration metrics typically rely on a binary true/false assessment of response correctness. This approach does not apply to long-form generation, where an answer can be partially correct. Addressing this gap, we introduce a unified calibration framework, in which both the correctness of the LLMs' responses and their associated confidence levels are treated as distributions across a range of scores. Within this framework, we develop three metrics to precisely evaluate LLM calibration and further propose two confidence elicitation methods based on self-consistency and self-evaluation. Our experiments, which include long-form QA and summarization tasks, demonstrate that larger models don't necessarily guarantee better calibration, that calibration performance is found to be metric-dependent, and that self-consistency methods excel in factoid datasets. We also find that calibration can be enhanced through techniques such as fine-tuning, integrating relevant source documents, scaling the temperature, and combining self-consistency with self-evaluation. Lastly, we showcase a practical application of our system: selecting and cascading open-source models and ChatGPT to optimize correctness given a limited API budget. This research not only challenges existing notions of LLM calibration but also offers practical methodologies for improving trustworthiness in long-form generation.
Calibrating Large Language Models Using Their Generations Only
Parameter Lab（参数实验室）；IT University of Copenhagen（哥本哈根信息技术大学）；Pioneer Centre for Artificial Intelligence（人工智能先锋中心）；NAVER AI Lab（NAVER人工智能实验室）；University of Tübingen（蒂宾根大学）；Tübingen AI Center（蒂宾根人工智能中心）
PsychePass: Calibrating LLM Therapeutic Competence via Trajectory-Anchored Tournaments
School of Computer Science and Engineering, Central South University（中南大学计算机科学与工程学院）；Lingxin AI（灵心AI）；South China Normal University（华南师范大学）；CoAI Group, DCST, IAI, BNRIST, Tsinghua University（清华人工智能研究院）
Fill In The Gaps: Model Calibration and Generalization with Synthetic Data
School of Computing and Augmented Intelligence, Arizona State University（亚利桑那州立大学计算与增强智能学院）；School of Mathematical and Natural Sciences, Arizona State University（亚利桑那州立大学数学与自然科学学院）
ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generation
School of Computer Science and Engineering, Northeastern University, China（东北大学计算机科学与工程学院）；Department of Computer Science and Technology, Institute for AI, Tsinghua University, China（清华大学人工智能研究院计算机科学与技术系）；Microsoft Research Asia, Beijing, China（微软亚洲研究院）；Language Technologies Institute, Carnegie Mellon University, United States（卡内基梅隆大学语言技术研究所）
BalanceRAG: Joint Risk Calibration for Cascaded Retrieval-Augmented Generation
Beihang University（北航）；Shenzhen Institute of Advanced Technology（深圳先进技术研究院）；Zhejiang University of Finance & Economics（浙江财经大学）；University of Electronic Science and Technology of China（电子科技大学）
Long and Diverse Text Generation with Planning-based Hierarchical Variational Model
Institute for Artificial Intelligence（人工智能研究院）；Beijing National Research Center for Information Science and Technology（北京信息科学与技术国家研究中心）；Tsinghua University（清华大学）；Baozun（宝尊）
LongEval: A Comprehensive Analysis of Long-Text Generation Through a Plan-based Paradigm
University of Manchester（曼彻斯特大学）；University of Surrey（萨里大学）；University of Sheffield（谢菲尔德大学）；Peking University（北京大学）；Hong Kong Polytechnic University（香港理工大学）
