Guaranteeing the correctness and factuality of language model (LM) outputs is a major open problem. In this work, we propose conformal factuality, a framework that can ensure high probability correctness guarantees for LMs by connecting language modeling and conformal prediction. We observe that the correctness of an LM output is equivalent to an uncertainty quantification problem, where the uncertainty sets are defined as the entailment set of an LM's output. Using this connection, we show that conformal prediction in language models corresponds to a back-off algorithm that provides high probability correctness guarantees by progressively making LM outputs less specific (and expanding the associated uncertainty sets). This approach applies to any black-box LM and requires very few human-annotated samples. Evaluations of our approach on closed book QA (FActScore, NaturalQuestions) and reasoning tasks (MATH) show that our approach can provide 80-90% correctness guarantees while retaining the majority of the LM's original output.
Differentiable Conformal Training for LLM Reasoning Factuality
Department of Computer Science, University of Illinois at Chicago, Chicago, United States(伊利诺伊大学芝加哥分校计算机科学系);Department of Physics, University of Illinois at Urbana-Champaign, Urbana, United States(伊利诺伊大学厄巴纳-香槟分校物理系)
CalPro: Prior-Aware Evidential--Conformal Prediction with Structure-Aware Guarantees for Protein Structures
Department of Computer Science, Iowa State University, Ames, Iowa, USA;Department of Civil, Construction \& Environmental Engineering, Iowa State University, Ames, Iowa, USA
Stochastic Grouping Conformal Prediction for Effective Subgroup Reliability
University of Electronic Science and Technology of China(电子科技大学);Kashi Institute of Electronics and Information Industry(喀什电子与信息产业研究院);Intelligent Digital Media Technology(智能数字媒体技术研究院)
Efficient Federated Conformal Prediction with Group-Conditional Guarantee
IoT Thrust(物联网 thrust);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Institute for Intelligent Networked Systems (INSI)(智能网络化系统研究所);Northeastern University London(伦敦东北大学);Department of ECE The Hong Kong University of Science and Technology HK SAR(电子工程系 香港科技大学香港特别行政区)
Department of Computer Science, New Jersey Institute of Technology(计算机科学系,新泽西理工学院);Department of Mathematical Sciences, New Jersey Institute of Technology(数学科学系,新泽西理工学院)
Enhancing Adversarial Robustness with Conformal Prediction: A Framework for Guaranteed Model Reliability
Huaiyin Institute of Technology, Huai'an, China(淮阴职业技术学院);Chengdu Research Institute, City University of Hong Kong, Chengdu, China(香港城市大学成都研究院);Saarland University, Saarbrücken, Germany(萨尔兰大学);Alpha Benito Research, Los Angeles, USA(Alpha Benito研究公司);City University of Hong Kong, Hong Kong, China(香港城市大学)
ConfHit: Conformal Generative Design with Oracle Free Guarantees
Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校Siebel计算与数据科学学院);Department of Statistics and Data Science, Wharton School, University of Pennysylvania(宾夕法尼亚大学沃顿商学院统计与数据科学系)
Conformal Prediction for Indoor Positioning with Correctness Coverage Guarantees
College of Electronic and Information Engineering, Southwest University(西南大学电子与信息工程学院);School of Automation, Chongqing University of Posts and Telecommunications(重庆邮电大学自动化学院)
Reliable Statistical Guarantees for Conformal Predictors with Small Datasets
ETSIAE-UPM - School of Aeronautics, Universidad Politécnica de Madrid(西班牙马德里理工大学航空航天学院);Institute for Cross-Disciplinary Physics and Complex Systems (IFISC), CSIC-UIB(交叉学科物理与复杂系统研究所(IFISC),CSIC-UIB);Center for Computational Simulation, Universidad Politécnica de Madrid, Campus de Montegancedo, Boadilla del Monte(计算模拟中心,马德里理工大学蒙特加森多校区,博阿迪利亚德尔蒙特)
Audited Conformal Prediction for Classification under Unknown Distribution Shift
Department of Data Sciences and Operations, University of Southern California(数据科学与运营系,南加州大学);AI Foundations, Capital One(Capital One人工智能基础);Department of Data Sciences and Operations, Thomas Lord Department of Computer Science, University of Southern California(数据科学与运营系,托马斯·劳德计算机科学系,南加州大学)
Towards Statistical Factuality Guarantee for Large Vision-Language Models
Vanderbilt University(范德堡大学);Vanderbilt University Medical Center(范德堡大学医学中心);Intuit(英图伊特公司);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Intuit AI Research(英图伊特人工智能研究院)
Statistical Guarantees in Synthetic Data through Conformal Adversarial Generation
WorkOnward Inc WOW AI Labs California(WorkOnward公司WOW AI实验室加州);Department of Computer Science California State University Long Beach(计算机科学系加州州立大学长滩分校);Department of Computer Science Emory University(计算机科学系埃默里大学)
ConformalNL2LTL: Translating Natural Language Instructions into Temporal Logic Formulas with Conformal Correctness Guarantees
Department of Electrical and Systems Engineering, Washington University in St Louis(华盛顿大学圣路易斯分校电子与系统工程系);Department of Computer Science, University of Southern California(南加州大学计算机科学系)
Non-Exchangeable Conformal Language Generation with Nearest Neighbors
IT University of Copenhagen(哥本哈根信息技术大学);Pioneer Centre for Artificial Intelligence(先锋人工智能中心);Instituto de Telecomunicações(电信研究所);Instituto Superior Técnico, Universidade de Lisboa(里斯本大学高等技术学院);Unbabel(昂巴贝尔公司)
Probabilistic Conformal Coverage Guarantees in Small-Data Settings
Berkeley Synchrotron Infrared Structural Biology Program(伯克利同步辐射红外结构生物学项目);Center for Advanced Mathematics in Energy Research Applications(能源研究应用高级数学中心);Molecular Biophysics and Integrated Bioimaging Division(分子生物物理与整合生物成像部门)