Adequate uncertainty representation and quantification have become imperative in various scientific disciplines, especially in machine learning and artificial intelligence. As an alternative to representing uncertainty via one single probability measure, we consider credal sets (convex sets of probability measures). The geometric representation of credal sets as $d$-dimensional polytopes implies a geometric intuition about (epistemic) uncertainty. In this paper, we show that the volume of the geometric representation of a credal set is a meaningful measure of epistemic uncertainty in the case of binary classification, but less so for multi-class classification. Our theoretical findings highlight the crucial role of specifying and employing uncertainty measures in machine learning in an appropriate way, and for being aware of possible pitfalls.
CISPA Helmholtz Center for Information Security（CISPA亥姆霍兹信息安全中心）；University College London（伦敦大学学院）；Inria（法国国家信息与自动化研究所）；Gatsby Computational Neuroscience Unit（盖茨计算神经科学单元）；University of Adelaide（阿德莱德大学）
Paderborn University（帕德博恩大学）；University of Munich (LMU)（慕尼黑大学）
The idea to distinguish and quantify two important types of uncertainty, often referred to as aleatoric and epistemic, has received increasing attention in machine learning research in the last couple of years. In this paper, we consider ensemble-based approaches to uncertainty quantification. Distinguishing between different types of uncertainty-aware learning algorithms, we specifically focus on Bayesian methods and approaches based on so-called credal sets, which naturally suggest themselves from an ensemble learning point of view. For both approaches, we address the question of how to quan…
Optimal Conformal Prediction under Epistemic Uncertainty
LMU Munich（慕尼黑莱茵姆大学）；CISPA（Cybersecurity Institute for Safer Applications）；LMU Munich MCML（慕尼黑莱茵姆大学MCML）；Ghent University（根特大学）；University of Cologne（科隆大学）；DFKI（德国达姆施塔特研究所）
Learning Credal Ensembles via Distributionally Robust Optimization
Department of Computer Science, KU Leuven, Belgium（比利时库勒万大学计算机科学系）；College of Computing and Data Science, Nanyang Technological University, Singapore（新加坡南洋理工大学计算与数据科学学院）；Department of Mechanical Engineering, KU Leuven, Belgium（比利时库勒万大学机械工程系）；School of Engineering, Computing and Mathematics, Oxford Brookes University, U.K.（英国奥德赛布鲁克斯大学工程、计算与数学学院）
Institute of Informatics, LMU Munich（慕尼黑大学信息学研究所）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）
A calibration test for evaluating set-based epistemic uncertainty representations
Department of Data Analysis and Mathematical Modeling（数据分析与数学建模系）；Ghent University（根特大学）；Department of Environment（环境系）；Department of Informatics（信息系）；Munich Center for Machine Learning（慕尼黑机器学习中心）；LMU Munich（慕尼黑大学）
Credal Wrapper of Model Averaging for Uncertainty Estimation in Classification
DistriNet, Department of Computer Science, KU Leuven, Belgium（DistriNet，计算机科学系，KU Leuven，比利时）；LMSD, Department of Mechanical Engineering, KU Leuven, Belgium（LMSD，机械工程系，KU Leuven，比利时）；School of Engineering, Computing and Mathematics, Oxford Brookes University, UK（工程、计算与数学学院，奥克斯福德布鲁克斯大学，英国）；M-Group, KU Leuven, Belgium（M-Group，KU Leuven，比利时）
Credal Large Language Models for Semantic Commitment under Uncertainty
Oxford Dynamics（牛津动力学）；Ludwig-Maximilians-Universität München（慕尼黑大学）；Institute for Artificial Intelligence, Data Analysis and Systems (AIDAS)（人工智能、数据分析与系统研究所）；School of Engineering Computing & Mathematics（工程计算与数学学院）；Oxford Brookes University（牛津布鲁克斯大学）
Credal and Interval Deep Evidential Classifications
Department of Computer Science, The University of Manchester（曼彻斯特大学计算机科学系）；School of Engineering, Computing and Mathematics, Oxford Brookes University（奥克斯布里奇大学工程、计算与数学学院）
Beyond Prediction-Set Size: Quantifying Conformal Uncertainty through Credal Geometry
Epistemic Intelligence & Computation Lab, College of Computing & Data Science, Nanyang Technological University, Singapore（新加坡南洋理工大学计算与数据科学学院）；CISPA Helmholtz Center for Information Security, Germany（德国亥姆霍兹信息安全中心）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Institute of Informatics, LMU Munich, Germany（德国慕尼黑机器学习中心）；Munich Center for Machine Learning, Germany（曼彻斯特人工智能基础研究中心）；Manchester Centre for AI Fundamentals, Manchester, United Kingdom Department of Computer Science, The University of Manchester, United Kingdom
Paderborn University（帕德博恩大学）；University of Munich (LMU)（慕尼黑大学）
Self-training is an effective approach to semi-supervised learning. The key idea is to let the learner itself iteratively generate "pseudo-supervision" for unlabeled instances based on its current hypothesis. In combination with consistency regularization, pseudo-labeling has shown promising performance in various domains, for example in computer vision. To account for the hypothetical nature of the pseudo-labels, these are commonly provided in the form of probability distributions. Still, one may argue that even a probability distribution represents an excessive level of informedness, as it s…
Sorbonne Universités（索邦大学联盟）；Université Technologique de Compiègne（贡比涅技术大学）；CNRS（法国国家科学研究中心）
We present two different strategies to extend the classical multi-label chaining approach to handle imprecise probability estimates. These estimates use convex sets of distributions (or credal sets) in order to describe our uncertainty rather than a precise one. The main reasons one could have for using such estimations are (1) to make cautious predictions (or no decision at all) when a high uncertainty is detected in the chaining and (2) to make better precise predictions by avoiding biases caused in early decisions in the chaining. We adapt both strategies to the case of the naive credal cla…
The University of Manchester（曼彻斯特大学）；University of British Columbia（不列颠哥伦比亚大学）；University of Pennsylvania（宾夕法尼亚大学）；Rensselaer Polytechnic Institute（伦斯勒理工学院）
Incentive Aware AI Regulations: A Credal Characterisation
CISPA Helmholtz Center for Information Security（CISPA亥姆霍兹信息安全中心）；Department of Statistics, LMU Munich（慕尼黑大学统计系）；UvA-Bosch Delta Lab, University of Amsterdam（阿姆斯特丹大学UvA-Bosch Delta实验室）；Nanyang Technological University（南洋理工大学）
How to efficiently explore in reinforcement learning is an open problem. Many exploration algorithms employ the epistemic uncertainty of their own value predictions -- for instance to compute an exploration bonus or upper confidence bound. Unfortunately the required uncertainty is difficult to estimate in general with function approximation. We propose epistemic value estimation (EVE): a recipe that is compatible with sequential decision making and with neural network function approximators. It equips agents with a tractable posterior over all their parameters from which epistemic value uncert…
Curricula for goal-conditioned reinforcement learning agents typically rely on poor estimates of the agent's epistemic uncertainty or fail to consider the agents' epistemic uncertainty altogether, resulting in poor sample efficiency. We propose a novel algorithm, Query The Agent (QTA), which significantly improves sample efficiency by estimating the agent's epistemic uncertainty throughout the state space and setting goals in highly uncertain areas. Encouraging the agent to collect data in highly uncertain states allows the agent to improve its estimation of the value function rapidly. QTA uti…
Mila - Quebec AI Institute（米拉-魁北克人工智能研究所）；Université de Montréal（蒙特利尔大学）；Massachusetts Institute of Technology（麻省理工学院）
Epistemic Uncertainty is a measure of the lack of knowledge of a learner which diminishes with more evidence. While existing work focuses on using the variance of the Bayesian posterior due to parameter uncertainty as a measure of epistemic uncertainty, we argue that this does not capture the part of lack of knowledge induced by model misspecification. We discuss how the excess risk, which is the gap between the generalization error of a predictor and the Bayes predictor, is a sound measure of epistemic uncertainty which captures the effect of model misspecification. We thus propose a principl…
Heinz Nixdorf Institute（海因茨·尼克斯多夫研究所）；Paderborn University（帕德博恩大学）；UMR CNRS 7253 Heudiasyc（法国国家科学研究中心联合研究单位7253 Heudiasyc）；Sorbonne Universités（索邦大学）；Université de Technologie de Compiègne（贡比涅技术大学）
Various strategies for active learning have been proposed in the machine learning literature. In uncertainty sampling, which is among the most popular approaches, the active learner sequentially queries the label of those instances for which its current prediction is maximally uncertain. The predictions as well as the measures used to quantify the degree of uncertainty, such as entropy, are almost exclusively of a probabilistic nature. In this paper, we advocate a distinction between two different types of uncertainty, referred to as epistemic and aleatoric, in the context of active learning. …
HybridFlow: Quantification of Aleatoric and Epistemic Uncertainty with a Single Hybrid Model
Department of Earth, Environment, and Planetary Sciences, Brown University（布朗大学地球、环境与行星科学系）；Data Science Institute, Brown University（布朗大学数据科学研究院）；Department of Computer Science, Brown University（布朗大学计算机科学系）
Technical University Munich（慕尼黑工业大学）；Autonomous Intelligent Driving GmbH（自主智能驾驶有限公司）；Google（谷歌）
We present a sampling-free approach for computing the epistemic uncertainty of a neural network. Epistemic uncertainty is an important quantity for the deployment of deep neural networks in safety-critical applications, since it represents how much one can trust predictions on new data. Recently promising works were proposed using noise injection combined with Monte-Carlo sampling at inference time to estimate this quantity (e.g. Monte-Carlo dropout). Our main contribution is an approximation of the epistemic uncertainty estimated by these methods that does not require sampling, thus notably r…
Due to the steadily increasing relevance of machine learning for practical applications, many of which are coming with safety requirements, the notion of uncertainty has received increasing attention in machine learning research in the last couple of years. In particular, the idea of distinguishing between two important types of uncertainty, often refereed to as aleatoric and epistemic, has recently been studied in the setting of supervised learning. In this paper, we propose to quantify these uncertainties with random forests. More specifically, we show how two general approaches for measurin…
University of Munich (LMU)（慕尼黑大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Ghent University（根特大学）
Uncertainty quantification has received increasing attention in machine learning in the recent past. In particular, a distinction between aleatoric and epistemic uncertainty has been found useful in this regard. The latter refers to the learner's (lack of) knowledge and appears to be especially difficult to measure and quantify. In this paper, we analyse a recent proposal based on the idea of a second-order learner, which yields predictions in the form of distributions over probability distributions. While standard (first-order) learners can be trained to predict accurate probabilities, namely…
The notion of uncertainty is of major importance in machine learning and constitutes a key element of machine learning methodology. In line with the statistical tradition, uncertainty has long been perceived as almost synonymous with standard probability and probabilistic predictions. Yet, due to the steadily increasing relevance of machine learning for practical applications and related issues such as safety requirements, new problems and challenges have recently been identified by machine learning scholars, and these problems may call for new methodological developments. In particular, this …
Kansas State University（堪萨斯州立大学）；ETH Zurich（苏黎世联邦理工学院）
Graph Neural Networks (GNN) provide a powerful framework that elegantly integrates Graph theory with Machine learning for modeling and analysis of networked data. We consider the problem of quantifying the uncertainty in predictions of GNN stemming from modeling errors and measurement uncertainty. We consider aleatoric uncertainty in the form of probabilistic links and noise in feature vector of nodes, while epistemic uncertainty is incorporated via a probability distribution over the model parameters. We propose a unified approach to treat both sources of uncertainty in a Bayesian framework, …
University of Munich (LMU)（慕尼黑大学（LMU））；Munich Center for Machine Learning（慕尼黑机器学习中心）；Ghent University（根特大学）
It is well known that accurate probabilistic predictors can be trained through empirical risk minimisation with proper scoring rules as loss functions. While such learners capture so-called aleatoric uncertainty of predictions, various machine learning methods have recently been developed with the goal to let the learner also represent its epistemic uncertainty, i.e., the uncertainty caused by a lack of knowledge and data. An emerging branch of the literature proposes the use of a second-order learner that provides predictions in terms of distributions on probability distributions. However, re…
