Obtaining heteroscedastic predictive uncertainties from a Bayesian Neural Network (BNN) is vital to many applications. Often, heteroscedastic aleatoric uncertainties are learned as outputs of the BNN in addition to the predictive means, however doing so may necessitate adding more learnable parameters to the network. In this work, we demonstrate that both the heteroscedastic aleatoric and epistemic variance can be embedded into the variances of learned BNN parameters, improving predictive performance for lightweight networks. By complementing this approach with a moment propagation approach to inference, we introduce a relatively simple framework for sampling-free variational inference suitable for lightweight BNNs.
Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）
The role of uncertainty quantification (UQ) in deep learning has become crucial with growing use of predictive models in high-risk applications. Though a large class of methods exists for measuring deep uncertainties, in practice, the resulting estimates are found to be poorly calibrated, thus making it challenging to translate them into actionable insights. A common workaround is to utilize a separate recalibration step, which adjusts the estimates to compensate for the miscalibration. Instead, we propose to repurpose the heteroscedastic regression objective as a surrogate for calibration and…
The estimation of model parameters with uncertainties from observed data is a ubiquitous inverse problem in science and engineering. In this paper, we suggest an inexpensive and easy to implement parameter estimation technique that uses a heteroscedastic Bayesian Neural Network trained using anchored ensembling. The heteroscedastic aleatoric error of the network models the irreducible uncertainty due to parameter degeneracies in our inverse problem, while the epistemic uncertainty of the Bayesian model captures uncertainties which may arise from an input observation's out-of-distribution natur…
Uncertainty-Aware Trajectory Prediction via Rule-Regularized Heteroscedastic Deep Classification
Department of Mathematics and Computer Science, Freie Universität Berlin（自由大学柏林数学与计算机科学系）；Continental Automotive Technologies GmbH, AI Lab Berlin（大陆汽车技术有限公司柏林AI实验室）；Karlsruhe Institute of Technology, Scientific Computing Center, Methods for Big Data（卡尔斯鲁厄理工学院科学计算中心、大数据方法）；Fraunhofer Institute for Open Communication Systems, Berlin, Germany（弗劳恩霍夫开放通信系统研究所，柏林德国）
Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）
Calibration error is commonly adopted for evaluating the quality of uncertainty estimators in deep neural networks. In this paper, we argue that such a metric is highly beneficial for training predictive models, even when we do not explicitly measure the uncertainties. This is conceptually similar to heteroscedastic neural networks that produce variance estimates for each prediction, with the key difference that we do not place a Gaussian prior on the predictions. We propose a novel algorithm that performs simultaneous interval estimation for different calibration levels and effectively levera…
Deep learning techniques have been shown to be extremely effective for various classification and regression problems, but quantifying the uncertainty of their predictions and separating them into the epistemic and aleatoric fractions is still considered challenging. In oil and gas exploration projects, tools consisting of seismic, sonic, magnetic resonance, resistivity, dielectric and/or nuclear sensors are sent downhole through boreholes to probe the earth's rock and fluid properties. The measurements from these tools are used to build reservoir models that are subsequently used for estimati…
Data-driven visual odometry (VO) is a critical subroutine for autonomous edge robotics, and recent progress in the field has produced highly accurate point predictions in complex environments. However, emerging autonomous edge robotics devices like insect-scale drones and surgical robots lack a computationally efficient framework to estimate VO's predictive uncertainties. Meanwhile, as edge robotics continue to proliferate into mission-critical application spaces, awareness of model's the predictive uncertainties has become crucial for risk-aware decision-making. This paper addresses this chal…
Massachusetts Institute of Technology（麻省理工学院）；BITS Pilani Goa Campus（BITS Pilani果阿校区）
Understanding and quantifying uncertainty in black box Neural Networks (NNs) is critical when deployed in real-world settings such as healthcare. Recent works using Bayesian and non-Bayesian methods have shown how a unified predictive uncertainty can be modelled for NNs. Decomposing this uncertainty to disentangle the granular sources of heteroscedasticity in data provides rich information about its underlying causes. We propose a conceptually simple non-Bayesian approach, deep split ensemble, to disentangle the predictive uncertainties using a multivariate Gaussian mixture model. The NNs are …
Cooperative Variance Estimation and Bayesian Neural Networks for Disentangling Aleatoric and Epistemic Uncertainties
Faculty of Mechanical Engineering, Delft University of Technology, Mekelweg 2, Delft, 2628 CD, The Netherlands（代尔夫特理工大学机械工程学院）；School of Engineering, Brown University, 184 Hope St., Providence, RI 02912, USA（布朗大学工程学院）
Improved Uncertainty Quantification in Physics-Informed Neural Networks Using Error Bounds and Solution Bundles
Departamento de Ciencia de la Computación, Pontificia Universidad Católica de Chile（计算机科学系，天主教智利大学）；Department of Computer Science, University of Tübingen（计算机科学系，图宾根大学）；John A. Paulson School of Engineering and Applied Sciences, Harvard University（工程与应用科学学院，哈佛大学）
Even though probabilistic treatments of neural networks have a long history, they have not found widespread use in practice. Sampling approaches are often too slow already for simple networks. The size of the inputs and the depth of typical CNN architectures in computer vision only compound this problem. Uncertainty in neural networks has thus been largely ignored in practice, despite the fact that it may provide important information about the reliability of predictions and the inner workings of the network. In this paper, we introduce two lightweight approaches to making supervised learning …
Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）
Techniques for understanding the functioning of complex machine learning models are becoming increasingly popular, not only to improve the validation process, but also to extract new insights about the data via exploratory analysis. Though a large class of such tools currently exists, most assume that predictions are point estimates and use a sensitivity analysis of these estimates to interpret the model. Using lightweight probabilistic networks we show how including prediction uncertainties in the sensitivity analysis leads to: (i) more robust and generalizable models; and (ii) a new approach…
University of Cambridge（剑桥大学）；Lancaster University（兰卡斯特大学）；British Antarctic Survey（英国南极调查局）
Ensembles of geophysical models improve projection accuracy and express uncertainties. We develop a novel data-driven ensembling strategy for combining geophysical models using Bayesian Neural Networks, which infers spatiotemporally varying model weights and bias while accounting for heteroscedastic uncertainties in the observations. This produces more accurate and uncertainty-aware projections without sacrificing interpretability. Applied to the prediction of total column ozone from an ensemble of 15 chemistry-climate models, we find that the Bayesian neural network ensemble (BayNNE) outperfo…
Active operator learning with predictive uncertainty quantification for partial differential equations
Sandia National Laboratories（桑迪亚国家实验室）；Department of Statistics（统计学系）；Data Science, Yale University（数据科学，耶鲁大学）；Environmental Engineering, Yale University（环境工程，耶鲁大学）；Department of Mathematics（数学系）；Department of Mechanical Engineering（机械工程系）
Residual Reweighted Conformal Prediction for Graph Neural Networks
Department of System Engineering, City University of Hong Kong, China（香港城市大学系统工程系）；Chengdu Research Institute, City University of Hong Kong, China（香港城市大学成都研究所）；Faculty of Electronic Information Engineering, Huaiyin Institute of Technology, China（淮阴科技学院电子信息工程学院）；Alpha Benito Research, USA（Alpha Benito研究公司）；Centre for Intelligent Systems, Royal Holloway, University of London, UK（伦敦大学皇家霍洛威学院智能系统中心）；Shenzhen People’s Hospital, China（深圳人民医院）
Hybrid Neural Network and Conventional Controller Approach for Robust Control of Highly Unstable Systems: Application to Tilt-Rotor Control
Advanced Research Lab for Control and Agricultural Robotics (Sharif AgRoLab)（控制与农业机器人高级研究实验室（谢尔生产大学AgRoLab））；Department of Mechanical Engineering, Sharif University of Technology, Tehran, Iran（技术大学机械工程系，德黑兰，伊朗）
