Beyond Prediction-Set Size: Quantifying Conformal Uncertainty through Credal Geometry
Epistemic Intelligence & Computation Lab, College of Computing & Data Science, Nanyang Technological University, Singapore（新加坡南洋理工大学计算与数据科学学院）；CISPA Helmholtz Center for Information Security, Germany（德国亥姆霍兹信息安全中心）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Institute of Informatics, LMU Munich, Germany（德国慕尼黑机器学习中心）；Munich Center for Machine Learning, Germany（曼彻斯特人工智能基础研究中心）；Manchester Centre for AI Fundamentals, Manchester, United Kingdom Department of Computer Science, The University of Manchester, United Kingdom
Siu Lun Chau, Soroush H. Zargarbashi, Yusuf Sale, Michele Caprio
Prediction-set size offers a coarse summary of conformal uncertainty. We develop a richer perspective through the geometry of credal sets: families of probability distributions compatible with conformal prediction (CP) regions. We establish this representation for split CP, extending a correspondence recently identified for full CP. Building on this representation, we derive analytical characterisations and exact algorithms for several widely used credal uncertainty measures, reducing potentially exponential computations to linear time after sorting the conformal p-values. We further unify several of these measures with classical conformal summaries by integrating set-size penalties based on Tsallis entropies across error levels. We also prove an impossibility theorem: for common split-conformal regression constructions, uncertainty summaries of the induced credal sets cannot distinguish between inputs. Classification experiments on selective prediction and distribution-shift detection show that these summaries often recover useful ranking information lost by prediction-set size at a fixed error level, enabling finer-grained uncertainty assessment. These results establish credal geometry as a principled framework for quantifying conformal uncertainty and understanding its limitations.
Distilling Calibration via Conformalized Credal Inference
1 King's Communications, Learning \& Information Processing (KCLIP) lab Centre for Intelligent Information Processing Systems (CIIPS), King’s College London, London, UK
The University of Manchester（曼彻斯特大学）；University of British Columbia（不列颠哥伦比亚大学）；University of Pennsylvania（宾夕法尼亚大学）；Rensselaer Polytechnic Institute（伦斯勒理工学院）
Institute of Informatics, LMU Munich（慕尼黑大学信息学研究所）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）
UMR CNRS 7253 Heudiasyc Université de Technologie de Compiègne France（法国克雷伊根技术大学7253 UMR CNRS Heudiasyc实验室）；Laboratoire de Génie Informatique et d’Automatique de l’Artois (LGI2A) Université d’Artois, UR 3926, France（阿托瓦大学自动化与计算机工程实验室（LGI2A））
Optimal Conformal Prediction under Epistemic Uncertainty
LMU Munich（慕尼黑莱茵姆大学）；CISPA（Cybersecurity Institute for Safer Applications）；LMU Munich MCML（慕尼黑莱茵姆大学MCML）；Ghent University（根特大学）；University of Cologne（科隆大学）；DFKI（德国达姆施塔特研究所）
Quantifying uncertainty in model predictions is a common goal for practitioners seeking more than just point predictions. One tool for uncertainty quantification that requires minimal assumptions is conformal inference, which can help create probabilistically valid prediction regions for black box models. Classical conformal prediction only provides marginal validity, whereas in many situations locally valid prediction regions are desirable. Deciding how best to partition the feature space X when applying localized conformal prediction is still an open question. We present MD-split+, a practic…
Division of Systems and Control, Department of Information Technology, Uppsala University, Sweden（信息科技系系统与控制 division，乌普萨拉大学，瑞典）；Science for Life Laboratory, Uppsala, Sweden（生命科学实验室，乌普萨拉，瑞典）
Uncertainty-Aware Multimodal Learning via Conformal Shapley Intervals
Department of Statistics and Data Science, UCLA（统计与数据科学系， UCLA）；Merck & Co（默克公司）；Department of Biostatistics, UCLA（生物统计学系， UCLA）；Departments of Statistics and Data Science, and of Biostatistics, UCLA（统计与数据科学系以及生物统计学系， UCLA）
The regulatory approval and broad clinical deployment of medical AI have been hampered by the perception that deep learning models fail in unpredictable and possibly catastrophic ways. A lack of statistically rigorous uncertainty quantification is a significant factor undermining trust in AI results. Recent developments in distribution-free uncertainty quantification present practical solutions for these issues by providing reliability guarantees for black-box models on arbitrary data distributions as formally valid finite-sample prediction intervals. Our work applies these new uncertainty qua…
A Systematic Review of Conformal Inference Procedures for Treatment Effect Estimation: Methods and Challenges
Engineering Mathematics and Computing Lab Heidelberg University（海德堡大学工程数学与计算实验室）；Heidelberg University（海德堡大学）；Heidelberg Institute for Theoretical Studies, Data Mining and Uncertainty Quantification Group（海德堡理论研究所数据挖掘与不确定性量化组）；Medical Faculty Mannheim of Heidelberg University, Data Analysis and Modeling in Medicine（海德堡大学曼海姆医学系医学数据分析与建模）
Uncertainty Quantification and Data Efficiency in AI: An Information-Theoretic Perspective
Institute for Intelligent Networked Systems, Northeastern University London（智能网络系统研究所，诺思安普顿大学伦敦分校）；Departments of Electrical and Computer Engineering and of Computer Science at the Technion–Israel Institute of Technology（技术ion-以色列理工学院电子与计算机工程系和计算机科学系）
Isotonic Quantile Regression Averaging for uncertainty quantification of electricity price forecasts
Doctoral School, Faculty of Management, Wrocław University of Science and Technology（沃斯克拉大学管理学院博士学院）；Department of Operations Research and Business Intelligence, Wrocław University of Science and Technology（沃斯克拉大学运筹学与商业智能系）
Distribution-informed Efficient Conformal Prediction for Full Ranking
Department of Mathematics, Chinese University of Hong Kong（中国香港大学数学系）；Department of Statistics and Data Science, Southern University of Science and Technology（南方科技大学统计与数据科学系）；School of Computation, Information, and Technology, Technical University of Munich（慕尼黑技术大学计算、信息与技术学院）
Conformalised Conditional Normalising Flows for Joint Prediction Regions in time series
Hasso Plattner Institute for Digital Engineering（哈索·普拉特纳数字工程学院）；University of California, Irvine（加利福尼亚大学欧文分校）；Hasso Plattner Institute for Digital Health at the Icahn School of Medicine at Mount Sinai（西奈山伊坎医学院哈索·普拉特纳数字健康研究所）
RIKEN Center for Advanced Intelligence Project（理化学研究所先进智能项目中心）；Nagoya Institute of Technology（名古屋工业大学）
If you are predicting the label $y$ of a new object with $\hat y$, how confident are you that $y = \hat y$? Conformal prediction methods provide an elegant framework for answering such question by building a $100 (1 - α)\%$ confidence region without assumptions on the distribution of the data. It is based on a refitting procedure that parses all the possibilities for $y$ to select the most likely ones. Although providing strong coverage guarantees, conformal set is impractical to compute exactly for many regression problems. We propose efficient algorithms to compute conformal prediction set u…
Non-Exchangeable Conformal Language Generation with Nearest Neighbors
IT University of Copenhagen（哥本哈根信息技术大学）；Pioneer Centre for Artificial Intelligence（先锋人工智能中心）；Instituto de Telecomunicações（电信研究所）；Instituto Superior Técnico, Universidade de Lisboa（里斯本大学高等技术学院）；Unbabel（昂巴贝尔公司）
Spatially-Adaptive Conformal Graph Transformer for Indoor Localization in Wi-Fi Driven Networks
Department of Computer Science, The University of Vermont（佛罗里达大学计算机科学系）；Department of Computer Science & Engineering, American University of Sharjah（阿曼大学计算机科学与工程系）
University of Cambridge（剑桥大学）；UCLA（加州大学洛杉矶分校）；The Alan Turing Institute（艾伦·图灵研究所）
Successful application of machine learning models to real-world prediction problems, e.g. financial forecasting and personalized medicine, has proved to be challenging, because such settings require limiting and quantifying the uncertainty in the model predictions, i.e. providing valid and accurate prediction intervals. Conformal Prediction is a distribution-free approach to construct valid prediction intervals in finite samples. However, the prediction intervals constructed by Conformal Prediction are often (because of over-fitting, inappropriate measures of nonconformity, or other issues) ov…
A typical desideratum for quantifying the uncertainty from a classification model as a prediction set is class-conditional singleton set calibration. That is, such sets should map to the output of well-calibrated selective classifiers, matching the observed frequencies of similar instances. Recent works proposing adaptive and localized conformal p-values for deep networks do not guarantee this behavior, nor do they achieve it empirically. Instead, we use the strong signals for prediction reliability from KNN-based approximations of Transformer networks to construct data-driven partitions for M…
