Probabilistic Label Spreading: Efficient and Consistent Estimation of Soft Labels with Epistemic Uncertainty on Graphs
Institute of Computer Science Osnabrück University（计算机科学研究所 奥斯纳布鲁克大学）；Department of Mathematics Technical University of Berlin（数学系 柏林技术大学）；Department of Mathematics University of Wuppertal（数学系 威斯巴赫大学）；Quality Match GmbH Heidelberg, Germany（Quality Match GmbH 海德堡德国）
Jonathan Klees, Tobias Riedlinger, Peter Stehr, Bennet Böddecker, Daniel Kondermann, Matthias Rottmann
Safe artificial intelligence for perception tasks remains a major challenge, partly due to the lack of data with high-quality labels. Annotations themselves are subject to aleatoric and epistemic uncertainty, which is typically ignored during annotation and evaluation. While crowdsourcing enables collecting multiple annotations per image to estimate these uncertainties, this approach is impractical at scale due to the required annotation effort. We introduce a probabilistic label spreading method that provides reliable estimates of aleatoric and epistemic uncertainty of labels. Assuming label smoothness over the feature space, we propagate single annotations using a graph-based diffusion method. We prove that label spreading yields consistent probability estimators even when the number of annotations per data point converges to zero. We present and analyze a scalable implementation of our method. Experimental results indicate that, compared to baselines, our approach substantially reduces the annotation budget required to achieve a desired label quality on common image datasets and achieves a new state of the art on the Data-Centric Image Classification benchmark.
HybridFlow: Quantification of Aleatoric and Epistemic Uncertainty with a Single Hybrid Model
Department of Earth, Environment, and Planetary Sciences, Brown University（布朗大学地球、环境与行星科学系）；Data Science Institute, Brown University（布朗大学数据科学研究院）；Department of Computer Science, Brown University（布朗大学计算机科学系）
Optimal Control of Probabilistic Dynamics Models via Mean Hamiltonian Minimization
Department of Machine Learning and Neural Computing, Donders Institute for Brain, Cognition and Behaviour, Radboud University（机器学习与神经计算系，多纳尔斯脑认知行为研究所，拉德堡德大学）；Cluster of Excellence Machine Learning, University of Tübingen（卓越机器学习集群，图宾根大学）；Department of Engineering, Control Group, University of Oxford（工程系，控制组，牛津大学）
ETH Zurich（苏黎世联邦理工学院）；LMU Munich（慕尼黑大学）；University of Zurich（苏黎世大学）
The ability to estimate epistemic uncertainty is often crucial when deploying machine learning in the real world, but modern methods often produce overconfident, uncalibrated uncertainty predictions. A common approach to quantify epistemic uncertainty, usable across a wide class of prediction models, is to train a model ensemble. In a naive implementation, the ensemble approach has high computational cost and high memory demand. This challenges in particular modern deep learning, where even a single deep network is already demanding in terms of compute and memory, and has given rise to a numbe…
Optimal Conformal Prediction under Epistemic Uncertainty
LMU Munich（慕尼黑莱茵姆大学）；CISPA（Cybersecurity Institute for Safer Applications）；LMU Munich MCML（慕尼黑莱茵姆大学MCML）；Ghent University（根特大学）；University of Cologne（科隆大学）；DFKI（德国达姆施塔特研究所）
Heinz Nixdorf Institute（海因茨·尼克斯多夫研究所）；Paderborn University（帕德博恩大学）；UMR CNRS 7253 Heudiasyc（法国国家科学研究中心联合研究单位7253 Heudiasyc）；Sorbonne Universités（索邦大学）；Université de Technologie de Compiègne（贡比涅技术大学）
Various strategies for active learning have been proposed in the machine learning literature. In uncertainty sampling, which is among the most popular approaches, the active learner sequentially queries the label of those instances for which its current prediction is maximally uncertain. The predictions as well as the measures used to quantify the degree of uncertainty, such as entropy, are almost exclusively of a probabilistic nature. In this paper, we advocate a distinction between two different types of uncertainty, referred to as epistemic and aleatoric, in the context of active learning. …
Variational LSTM with Augmented Inputs: Nonlinear Response History Metamodeling with Aleatoric and Epistemic Uncertainty
Department of Civil, Environmental, and Construction Engineering, Texas Tech University（德克萨斯理工大学土木、环境与建筑工程系）；Department of Civil and Environmental Engineering, Rensselaer Polytechnic Institute（伦斯勒理工学院土木与环境工程系）；National Wind Institute, Texas Tech University（德克萨斯理工大学国家风能研究所）
The notion of uncertainty is of major importance in machine learning and constitutes a key element of machine learning methodology. In line with the statistical tradition, uncertainty has long been perceived as almost synonymous with standard probability and probabilistic predictions. Yet, due to the steadily increasing relevance of machine learning for practical applications and related issues such as safety requirements, new problems and challenges have recently been identified by machine learning scholars, and these problems may call for new methodological developments. In particular, this …
Mila - Quebec AI Institute（米拉-魁北克人工智能研究所）；Université de Montréal（蒙特利尔大学）；Massachusetts Institute of Technology（麻省理工学院）
Epistemic Uncertainty is a measure of the lack of knowledge of a learner which diminishes with more evidence. While existing work focuses on using the variance of the Bayesian posterior due to parameter uncertainty as a measure of epistemic uncertainty, we argue that this does not capture the part of lack of knowledge induced by model misspecification. We discuss how the excess risk, which is the gap between the generalization error of a predictor and the Bayes predictor, is a sound measure of epistemic uncertainty which captures the effect of model misspecification. We thus propose a principl…
University of Munich (LMU)（慕尼黑大学（LMU））；Munich Center for Machine Learning（慕尼黑机器学习中心）；Ghent University（根特大学）
It is well known that accurate probabilistic predictors can be trained through empirical risk minimisation with proper scoring rules as loss functions. While such learners capture so-called aleatoric uncertainty of predictions, various machine learning methods have recently been developed with the goal to let the learner also represent its epistemic uncertainty, i.e., the uncertainty caused by a lack of knowledge and data. An emerging branch of the literature proposes the use of a second-order learner that provides predictions in terms of distributions on probability distributions. However, re…
Kansas State University（堪萨斯州立大学）；ETH Zurich（苏黎世联邦理工学院）
Graph Neural Networks (GNN) provide a powerful framework that elegantly integrates Graph theory with Machine learning for modeling and analysis of networked data. We consider the problem of quantifying the uncertainty in predictions of GNN stemming from modeling errors and measurement uncertainty. We consider aleatoric uncertainty in the form of probabilistic links and noise in feature vector of nodes, while epistemic uncertainty is incorporated via a probability distribution over the model parameters. We propose a unified approach to treat both sources of uncertainty in a Bayesian framework, …
Breaking the Chains of Probability: Neutrosophic Logic as a New Framework for Epistemic Uncertainty in Large Language Models
Universidad Bolivariana del Ecuador, Coordinación Académica de Posgrado（巴尔干大学厄瓜多尔分校，研究生院）；Universidad de Guayaquil（瓜亚基尔大学）；Universidad Bernardo O’Higgins（伯纳多·奥希金斯大学）；Mathematics, Physics, and Natural Sciences Division, University of New Mexico（新墨西哥大学数学、物理和自然科学系）
Instituto de Ingenieria de Aragon (I3A)（阿拉贡工程研究所）；Universidad de Zaragoza（萨拉戈萨大学）
Affordances are a fundamental concept in robotics since they relate available actions for an agent depending on its sensory-motor capabilities and the environment. We present a novel Bayesian deep network to detect affordances in images, at the same time that we quantify the distribution of the aleatoric and epistemic variance at the spatial level. We adapt the Mask-RCNN architecture to learn a probabilistic representation using Monte Carlo dropout. Our results outperform the state-of-the-art of deterministic networks. We attribute this improvement to a better probabilistic feature space repre…
Neuronal Stochastic Attention Circuit (NSAC) for Probabilistic Representation Learning
Department of Automation, University of Science \& Technology of China, Hefei, China；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center
Comparison of Deterministic and Probabilistic Machine Learning Algorithms for Precise Dimensional Control and Uncertainty Quantification in Additive Manufacturing
Center for Nanoscale Materials, Argonne National Laboratory（阿贡国家实验室纳米尺度材料中心）；Department of Mechanical and Industrial Engineering, University of Illinois, Chicago（伊利诺伊大学芝加哥分校机械与工业工程系）；Shell International Exploration and Production Inc.（壳牌国际勘探与生产公司）
National Institute of Technology Durgapur（杜尔加布尔国立技术学院）；Indian Institute of Technology Kharagpur（印度理工学院卡拉格普尔分校）
COVID-19 pandemic is severely impacting the lives of billions across the globe. Even after taking massive protective measures like nation-wide lockdowns, discontinuation of international flight services, rigorous testing etc., the infection spreading is still growing steadily, causing thousands of deaths and serious socio-economic crisis. Thus, the identification of the major factors of this infection spreading dynamics is becoming crucial to minimize impact and lifetime of COVID-19 and any future pandemic. In this work, a probabilistic cellular automata based method has been employed to model…
Michigan State University（密歇根州立大学）；Argonne National Laboratory（阿贡国家实验室）；Brookhaven National Laboratory（布鲁克海文国家实验室）
Robust machine learning models with accurately calibrated uncertainties are crucial for safety-critical applications. Probabilistic machine learning and especially the Bayesian formalism provide a systematic framework to incorporate robustness through the distributional estimates and reason about uncertainty. Recent works have shown that approximate inference approaches that take the weight space uncertainty of neural networks to generate ensemble prediction are the state-of-the-art. However, architecture choices have mostly been ad hoc, which essentially ignores the epistemic uncertainty from…
Epistemic Intelligence & Computation Lab, College of Computing & Data Science, Nanyang Technological University, Singapore（新加坡南洋理工大学计算与数据科学学院认知智能与计算实验室）；Department of Computer Science, The University of Manchester, United Kingdom（英国曼彻斯特大学计算机科学系）；Manchester Centre for AI Fundamentals, Manchester, United Kingdom（英国曼彻斯特人工智能基础研究中心）；Rational Intelligence Lab, CISPA Helmholtz Center for Information Security, Germany（德国CISPA海德堡信息安全中心理性智能实验室）
A calibration test for evaluating set-based epistemic uncertainty representations
Department of Data Analysis and Mathematical Modeling（数据分析与数学建模系）；Ghent University（根特大学）；Department of Environment（环境系）；Department of Informatics（信息系）；Munich Center for Machine Learning（慕尼黑机器学习中心）；LMU Munich（慕尼黑大学）
Bayesian Transformer for Probabilistic Load Forecasting in Smart Grids
O&M Analytics, AES Clean Energ（O&M分析部，AES清洁能源）；The AES Corporation（AES公司）；Department of Information and Communication Engineering（信息与通信工程系）；Pabna University of Science and Technology（帕纳大学科学与技术）
Disentangling Aleatoric and Epistemic Uncertainty in Physics-Informed Neural Networks. Application to Insulation Material Degradation Prognostics
Department of Electronics and Computer Science（电子与计算机科学系）；Mondragon University（蒙得龙大学）；Research & Development（研发）；CARTIF Technology Centre（CARTIF技术中心）；Department of Physical Chemistry（物理化学系）；University of the Basque Country (EHU)（巴斯克国家大学（EHU））；Basque Centre for Applied Mathematics (BCAM)（巴斯克应用数学中心（BCAM））；Department of Computer Science and Artificial Intelligence（计算机科学与人工智能系）
Uncertainty Quantification in Probabilistic Machine Learning Models: Theory, Methods, and Insights
Weill Cornell Medicine, Cornell University, New York, NY（韦尔·科恩医学中心，康奈尔大学，纽约，纽约）；Department of Electrical and Computer Engineering, Stony Brook University, Stony Brook, NY（电气与计算机工程系，史泰文斯布鲁克大学，史泰文斯布鲁克，纽约）
From Aleatoric to Epistemic: Exploring Uncertainty Quantification Techniques in Artificial Intelligence
University of Liverpool（利物浦大学）；University of Edinburgh（爱丁堡大学）；The University of Texas at Dallas（德州大学达拉斯分校）；Georgia Institute of Technology（佐治亚理工学院）；Emory University（埃默里大学）；Simon Fraser University（西蒙菲莎大学）；Kyoto University（京都大学）；Zhejiang University（浙江大学）；National Taiwan Normal University（国立台湾师范大学）；Indiana University（印第安纳大学）；Purdue University（普渡大学）；AppCubic（AppCubic公司）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；The Hong Kong University of Science and Technology（香港科技大学）
Each year a growing number of wind farms are being added to power grids to generate electricity. The power curve of a wind turbine, which exhibits the relationship between generated power and wind speed, plays a major role in assessing the performance of a wind farm. Neural networks have been used for power curve estimation. However, they do not produce a confidence measure for their output, unless computationally prohibitive Bayesian methods are used. In this paper, a probabilistic neural network with Monte Carlo dropout is considered to quantify the model (epistemic) uncertainty of the power…
George Mason University（乔治梅森大学）；Trinity College Dublin（都柏林圣三一学院）；Dublin Institute for Advanced Studies（都柏林高等研究院）；University of Oxford（牛津大学）；Center for Mathematics and Computer Science (CWI)（数学与计算机科学中心（CWI））；Stanford University（斯坦福大学）；Element AI；Southwest Research Institute（西南研究院）
Machine learning techniques have been successfully applied to super-resolution tasks on natural images where visually pleasing results are sufficient. However in many scientific domains this is not adequate and estimations of errors and uncertainties are crucial. To address this issue we propose a Bayesian framework that decomposes uncertainties into epistemic and aleatoric uncertainties. We test the validity of our approach by super-resolving images of the Sun's magnetic field and by generating maps measuring the range of possible high resolution explanations compatible with a given low resol…
Bayesian Neural Networks vs. Mixture Density Networks: Theoretical and Empirical Insights for Uncertainty-Aware Nonlinear Modeling
Department of Mathematics and Statistics, Bowling Green State University（数学与统计学系，布恩维尔格林州立大学）；Department of Biostatistics, University of Pennsylvania（生物统计学系，宾夕法尼亚大学）
The application of machine learning (ML) techniques in wireless communication domain has seen a tremendous growth over the years especially in the wireless sensing domain. However, the questions surrounding the ML model's inference reliability, and uncertainty associated with its predictions are never answered or communicated properly. This itself raises a lot of questions on the transparency of these ML systems. Developing ML systems with probabilistic modeling can solve this problem easily, where one can quantify uncertainty whether it is arising from the data (irreducible error or aleotoric…
It is an effective strategy for the multi-person pose tracking task in videos to employ prediction and pose matching in a frame-by-frame manner. For this type of approach, uncertainty-aware modeling is essential because precise prediction is impossible. However, previous studies have relied on only a single prediction without incorporating uncertainty, which can cause critical tracking errors if the prediction is unreliable. This paper proposes an extension to this approach with Sequential Monte Carlo (SMC). This naturally reformulates the tracking scheme to handle multiple predictions (or hyp…
Neural networks are increasingly being used in a variety of settings to predict wind direction and speed, two of the most important factors for estimating the potential power output of a wind farm. However, these predictions are arguably of limited value because classical neural networks lack the ability to express uncertainty. Here we instead consider the use of Bayesian Neural Networks (BNNs), for which the weights, biases and outputs are distributions rather than deterministic point values. This allows for the evaluation of both epistemic and aleatoric uncertainty and leads to well-calibrat…
