Uncertainty quantification (UQ) in natural language generation (NLG) tasks remains an open challenge, exacerbated by the closed-source nature of the latest large language models (LLMs). This study investigates applying conformal prediction (CP), which can transform any heuristic uncertainty notion into rigorous prediction sets, to black-box LLMs in open-ended NLG tasks. We introduce a novel uncertainty measure based on self-consistency theory, and then develop a conformal uncertainty criterion by integrating the uncertainty condition aligned with correctness into the CP algorithm. Empirical evaluations indicate that our uncertainty measure outperforms prior state-of-the-art methods. Furthermore, we achieve strict control over the correctness coverage rate utilizing 7 popular LLMs on 4 free-form NLG datasets, spanning general-purpose and medical scenarios. Additionally, the calibrated prediction sets with small size further highlights the efficiency of our method in providing trustworthy guarantees for practical open-ended NLG applications.
Conformal Prediction for Indoor Positioning with Correctness Coverage Guarantees
College of Electronic and Information Engineering, Southwest University(西南大学电子与信息工程学院);School of Automation, Chongqing University of Posts and Telecommunications(重庆邮电大学自动化学院)
Aligning Language Models with Selective Prediction
Department of Computer Science and Engineering(计算机科学与工程系);Bioinformatics and Computational Biology Program(生物信息学与计算生物学项目);Division of Biostatistics and Health Data Science(生物统计学与健康数据科学部);University of Minnesota Twin Cities(明尼苏达大学双城分校)
Beyond Correctness: Benchmarking Multi-dimensional Code Generation for Large Language Models
University of Chinese Academy of Sciences(中国科学院大学);Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所);Chinese Information Processing Laboratory(中文信息处理实验室);State Key Laboratory of Computer Science(计算机科学国家重点实验室);Key Laboratory of System Software, Chinese Academy of Sciences(中国科学院系统软件重点实验室)
Correctness Coverage Evaluation for Medical Multiple-Choice Question Answering Based on the Enhanced Conformal Prediction Framework
School of Electronic and Information Engineering, Tongji University(同济大学电子与信息工程学院);School of Electrical and Information Engineering, Hunan University(湖南大学电气与信息工程学院);School of Economics and Management, Fuzhou University(福州市经济学与管理学院)
Trusted Uncertainty in Large Language Models: A Unified Framework for Confidence Calibration and Risk-Controlled Refusal
University of Liechtenstein(列支敦士登大学);University of the Republic of San Marino(圣马尔科共和国大学);University of Mauritius(毛里求斯大学);International University of Monaco(摩纳哥国际大学);University of Andorra(安道尔大学);University of Gibraltar(直布罗陀大学);University of the Faroe Islands(法罗群岛大学);Ilisimatusarfik (University of Greenland)(格陵兰岛研究所(格陵兰大学));University of Prizren(普里兹伦大学)