With the increased use of data-driven approaches and machine learning-based methods in material science, the importance of reliable uncertainty quantification (UQ) of the predicted variables for informed decision-making cannot be overstated. UQ in material property prediction poses unique challenges, including the multi-scale and multi-physics nature of advanced materials, intricate interactions between numerous factors, limited availability of large curated datasets for model training, etc. Recently, Bayesian Neural Networks (BNNs) have emerged as a promising approach for UQ, offering a probabilistic framework for capturing uncertainties within neural networks. In this work, we introduce an approach for UQ within physics-informed BNNs, which integrates knowledge from governing laws in material modeling to guide the models toward physically consistent predictions. To evaluate the effectiveness of this approach, we present case studies for predicting the creep rupture life of steel alloys. Experimental validation with three datasets of collected measurements from creep tests demonstrates the ability of BNNs to produce accurate point and uncertainty estimates that are competitive or exceed the performance of the conventional method of Gaussian Process Regression. Similarly, we evaluated the suitability of BNNs for UQ in an active learning application and reported competitive performance. The most promising framework for creep life prediction is BNNs based on Markov Chain Monte Carlo approximation of the posterior distribution of network parameters, as it provided more reliable results in comparison to BNNs based on variational inference approximation or related NNs with probabilistic outputs. The codes are available at: https://github.com/avakanski/Creep-uncertainty-quantification.
We propose a deep learning methodology for multivariate regression that is based on pattern recognition that triggers fast learning over sensor data. We used a conversion of sensors-to-image which enables us to take advantage of Computer Vision architectures and training processes. In addition to this data preparation methodology, we explore the use of state-of-the-art architectures to generate regression outputs to predict agricultural crop continuous yield information. Finally, we compare with some of the top models reported in MLCAS2021. We found that using a straightforward training proces…
Lehigh University（理海大学）；Sandia National Laboratories（桑迪亚国家实验室）
We consider the problem of modeling heterogeneous materials where micro-scale dynamics and interactions affect global behavior. In the presence of heterogeneities in material microstructure it is often impractical, if not impossible, to provide quantitative characterization of material response. The goal of this work is to develop a Bayesian framework for uncertainty quantification (UQ) in material response prediction when using nonlocal models. Our approach combines the nonlocal operator regression (NOR) technique and Bayesian inference. Specifically, we use a Markov chain Monte Carlo (MCMC) …
We propose a framework for the assessment of uncertainty quantification in deep regression. The framework is based on regression problems where the regression function is a linear combination of nonlinear functions. Basically, any level of complexity can be realized through the choice of the nonlinear functions and the dimensionality of their domain. Results of an uncertainty quantification for deep regression are compared against those obtained by a statistical reference method. The reference method utilizes knowledge of the underlying nonlinear functions and is based on a Bayesian linear reg…
Uncertainty Quantification for Regression using Proper Scoring Rules
Department of Machine Learning, MBZUAI（机器学习部门，MBZUAI）；ELLIS Unit Linz and LIT AI Lab, Institute for Machine Learning, Johannes Kepler University Linz（林兹ELLIS单元和LIT AI实验室，机器学习研究所，林兹约翰内斯·开普勒大学）
Uncertainty Quantification for Deep Regression using Contextualised Normalizing Flows
Arquimea Research Center（阿奎米亚研究中心）；Department of Computer Science, University of York（约克大学计算机科学系）；Department of Elect. Eng., and Computer Science and Eng., Cyprus University of Technology（塞浦路斯技术大学电子工程与计算机科学系）
Uncertainty Quantification for Regression: A Unified Framework based on kernel scores
LMU Munich（慕尼黑莱茵恩大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；DLR-German Aerospace Center University of Tromsø（德国航空航天中心-特罗姆瑟大学）；German Research Center for Artificial Intelligence (DFKI, DSA)（德国人工智能研究中心）
Politecnico di Milano（米兰理工大学）；Beihang University（北京航空航天大学）
Online damage quantification suffers from insufficient labeled data that weakens its accuracy. In this context, adopting the domain adaptation on historical labeled data from similar structures/damages or simulated digital twin data to assist the current diagnosis task would be beneficial. However, most domain adaptation methods are designed for classification and cannot efficiently address damage quantification, a regression problem with continuous real-valued labels. This study first proposes a novel domain adaptation method, the Online Fuzzy-set-based Joint Distribution Adaptation for Regre…
The ever-growing size of the datasets renders well-studied learning techniques, such as Kernel Ridge Regression, inapplicable, posing a serious computational challenge. Divide-and-conquer is a common remedy, suggesting to split the dataset into disjoint partitions, obtain the local estimates and average them, it allows to scale-up an otherwise ineffective base approach. In the current study we suggest a fully data-driven approach to quantify uncertainty of the averaged estimator. Namely, we construct simultaneous element-wise confidence bands for the predictions yielded by the averaged estimat…
Paris-Saclay University（巴黎萨克雷大学）；ENSTA Paris（巴黎高等先进技术学院）；Institut Polytechnique de Paris（巴黎理工学院）
Monocular depth is important in many tasks, such as 3D reconstruction and autonomous driving. Deep learning based models achieve state-of-the-art performance in this field. A set of novel approaches for estimating monocular depth consists of transforming the regression task into a classification one. However, there is a lack of detailed descriptions and comparisons for Classification Approaches for Regression (CAR) in the community and no in-depth exploration of their potential for uncertainty estimation. To this end, this paper will introduce a taxonomy and summary of CAR approaches, a new un…
Increasingly high-stakes decisions are made using neural networks in order to make predictions. Specifically, meteorologists and hedge funds apply these techniques to time series data. When it comes to prediction, there are certain limitations for machine learning models (such as lack of expressiveness, vulnerability of domain shifts and overconfidence) which can be solved using uncertainty estimation. There is a set of expectations regarding how uncertainty should ``behave". For instance, a wider prediction horizon should lead to more uncertainty or the model's confidence should be proportion…
An Axiomatic Assessment of Entropy- and Variance-based Uncertainty Quantification in Regression
LMU Munich（慕尼黑莱布尼茨大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；DLR-German Aerospace Center University of Tromsø（德国航空航天中心图勒大学）；German Research Center for Artificial Intelligence (DFKI, DSA)（德国人工智能研究中心）
While several methods for predicting uncertainty on deep networks have been recently proposed, they do not readily translate to large and complex datasets. In this paper we utilize a simplified form of the Mixture Density Networks (MDNs) to produce a one-shot approach to quantify uncertainty in regression problems. We show that our uncertainty bounds are on-par or better than other reported existing methods. When applied to standard regression benchmark datasets, we show an improvement in predictive log-likelihood and root-mean-square-error when compared to existing state-of-the-art methods. W…
University of California, Berkeley（加州大学伯克利分校）；Technion - Israel Institute of Technology（以色列理工学院）
Image-to-image regression is an important learning task, used frequently in biological imaging. Current algorithms, however, do not generally offer statistical guarantees that protect against a model's mistakes and hallucinations. To address this, we develop uncertainty quantification techniques with rigorous statistical guarantees for image-to-image regression problems. In particular, we show how to derive uncertainty intervals around each pixel that are guaranteed to contain the true value with a user-specified confidence probability. Our methods work in conjunction with any base machine lea…
University of Pennsylvania（宾夕法尼亚大学）；Purdue University（普渡大学）；University of Michigan（密歇根大学）
Deep neural network (DNN) models have achieved state-of-the-art predictive accuracy in a wide range of supervised learning applications. However, accurately quantifying the uncertainty in DNN predictions remains a challenging task. For continuous outcome variables, an even more difficult problem is to estimate the predictive density function, which not only provides a natural quantification of the predictive uncertainty, but also fully captures the random variation in the outcome. In this work, we propose the Bayesian Deep Noise Neural Network (B-DeepNoise), which generalizes standard Bayesian…
IB-UQ: Information bottleneck based uncertainty quantification for neural function regression and neural operator learning
Shanghai Normal University（上海师范大学）；Shanghai Jiaotong University（上海交通大学）；Tongji University（同济大学）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）
Enhancing Uncertainty Quantification in Drug Discovery with Censored Regression Labels
AstraZeneca R&D（阿斯利康研发部）；ELLIS Unit Linz（林茨ELLIS单元）；Institute for Machine Learning（机器学习研究所）；Johannes Kepler University Linz（约翰内斯·开普勒大学林茨分校）；KU Leuven（鲁汶大学）；Chalmers University of Technology（查尔姆斯理工大学）
Among the many ways of quantifying uncertainty in a regression setting, specifying the full quantile function is attractive, as quantiles are amenable to interpretation and evaluation. A model that predicts the true conditional quantiles for each input, at all quantile levels, presents a correct and efficient representation of the underlying uncertainty. To achieve this, many current quantile-based methods focus on optimizing the so-called pinball loss. However, this loss restricts the scope of applicable regression models, limits the ability to target many desirable properties (e.g. calibrati…
Isotonic Quantile Regression Averaging for uncertainty quantification of electricity price forecasts
Doctoral School, Faculty of Management, Wrocław University of Science and Technology（沃斯克拉大学管理学院博士学院）；Department of Operations Research and Business Intelligence, Wrocław University of Science and Technology（沃斯克拉大学运筹学与商业智能系）
University of California, Santa Barbara（加州大学圣巴巴拉分校）
Fabrication process variations can significantly influence the performance and yield of nano-scale electronic and photonic circuits. Stochastic spectral methods have achieved great success in quantifying the impact of process variations, but they suffer from the curse of dimensionality. Recently, low-rank tensor methods have been developed to mitigate this issue, but two fundamental challenges remain open: how to automatically determine the tensor rank and how to adaptively pick the informative simulation samples. This paper proposes a novel tensor regression method to address these two challe…
Massachusetts Institute of Technology（麻省理工学院）；Technion – Israel Institute of Technology（以色列理工学院）
Various engineering systems such as naval and aerial vehicles, offshore structures, and mechanical components of motorized systems, are exposed to fatigue failures due to stochastic loadings. Methods for early failure prediction are essential for engineering, military, and civil applications. In addition to the prediction of time to failure (TtF), uncertainty quantification (UQ) is of major importance for real-time decision-making purposes. Usually, time domain or frequency domain methods are used for fatigue prediction, such as rainflow counting and Miner's rule or Dirlik's method. However, t…
The geographically weighted regression (GWR) is an essential tool for estimating the spatial variation of relationships between dependent and independent variables in geographical contexts. However, GWR suffers from the problem that classical linear regressions, which compose the GWR model, are more prone to be underfitting, especially for significant volume and complex nonlinear data, causing inferior comparative performance. Nevertheless, some advanced models, such as the decision tree and the support vector machine, can learn features from complex data more effectively while they cannot pro…
Improved probabilistic regression using diffusion models
Ludwig-Maximilians-Universität München（慕尼黑莱布尼茨大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；University of Tromsø（特罗姆斯大学）；DLR-German Aerospace Center（德国航空航天中心）
Purpose: Arterial Spin Labeling (ASL) is a quantitative, non-invasive alternative to perfusion imaging with contrast agents. Fixing values of certain model parameters in traditional ASL, which actually vary from region to region, may introduce bias in perfusion estimates. Adopting Magnetic Resonance Fingerprinting (MRF) for ASL is an alternative where these parameters are estimated alongside perfusion, but multiparametric estimation can degrade precision. We aim to improve the sensitivity of ASL-MRF signals to underlying parameters to counter this problem, and provide precise estimates. We als…
Spectroscopy Analysis with Machine Learning Regression for the Quantification of Carbon and Nitrogen Contents in Inceptisol and Oxisol Soil Types: Comparing Different Preprocessing and Validation methods as well as Feature Importance
Graduate Program in Electrical and Computer Engineering, Federal University of Technology - Paraná（巴拉那联邦科技大学电气与计算机工程研究生项目）；Academic Department of Chemistry, Federal University of Technology-Paraná（巴拉那联邦科技大学化学系）；Graduate Program in Production and Systems Engineering, Federal University of Technology-Paraná（巴拉那联邦科技大学生产与系统工程研究生项目）
University of Michigan（密歇根大学）；Catalysis Science and Technology Institute, University of Michigan（密歇根大学催化科学与技术研究所）
It is critical that machine learning (ML) model predictions be trustworthy for high-throughput catalyst discovery approaches. Uncertainty quantification (UQ) methods allow estimation of the trustworthiness of an ML model, but these methods have not been well explored in the field of heterogeneous catalysis. Herein, we investigate different UQ methods applied to a crystal graph convolutional neural network (CGCNN) to predict adsorption energies of molecules on alloys from the Open Catalyst 2020 (OC20) dataset, the largest existing heterogeneous catalyst dataset. We apply three UQ methods to the…
University of Pennsylvania（宾夕法尼亚大学）；Medical College of Wisconsin（威斯康星医学院）；Arizona State University（亚利桑那州立大学）
Bayesian Additive Regression Trees (BART) is a fully Bayesian approach to modeling with ensembles of trees. BART can uncover complex regression functions with high dimensional regressors in a fairly automatic way and provide Bayesian quantification of the uncertainty through the posterior. However, BART assumes IID normal errors. This strong parametric assumption can lead to misleading inference and uncertainty quantification. In this paper, we use the classic Dirichlet process mixture (DPM) mechanism to nonparametrically model the error distribution. A key strength of BART is that default pri…
Model-Free Kernel Conformal Depth Measures Algorithm for Uncertainty Quantification in Regression Models in Separable Hilbert Spaces
Department of Biostatistics（生物统计学系）；Harvard University（哈佛大学）；Department of Epidemiology and Biostatistics（流行病学与生物统计学系）；University of South Carolina（南卡罗来纳大学）；LTCI Telecom Paris Institut Polytechnique de Paris（巴黎电信理工学院LTCI）；UCLA（加州大学洛杉矶分校）
Harvard Medical School（哈佛医学院）；Google Research（谷歌研究院）；Harvard School of Public Health（哈佛公共卫生学院）
Uncertainty quantification for complex deep learning models is increasingly important as these techniques see growing use in high-stakes, real-world settings. Currently, the quality of a model's uncertainty is evaluated using point-prediction metrics such as negative log-likelihood or the Brier score on heldout data. In this study, we provide the first large scale evaluation of the empirical frequentist coverage properties of well known uncertainty quantification techniques on a suite of regression and classification tasks. We find that, in general, some methods do achieve desirable coverage p…
University Hospital Magdeburg（马格德堡大学医院）；Otto von Guericke University（奥托·冯·格里克大学）；Human Technopole（人类技术中心）；University of L’Aquila（拉奎拉大学）；Center for Behavioral Brain Sciences（行为脑科学中心）；German Center for Neurodegenerative Disease（德国神经退行性疾病中心）
Motion artefacts in magnetic resonance brain images can have a strong impact on diagnostic confidence. The assessment of MR image quality is fundamental before proceeding with the clinical diagnosis. Motion artefacts can alter the delineation of structures such as the brain, lesions or tumours and may require a repeat scan. Otherwise, an inaccurate (e.g. correct pathology but wrong severity) or incorrect diagnosis (e.g. wrong pathology) may occur. "\textit{Image quality assessment}" as a fast, automated step right after scanning can assist in deciding if the acquired images are diagnostically …
Quantile Regression, Variational Autoencoders, and Diffusion Models for Uncertainty Quantification: A Spatial Analysis of Sub-seasonal Wind Speed Prediction
HybridFlow: Quantification of Aleatoric and Epistemic Uncertainty with a Single Hybrid Model
Department of Earth, Environment, and Planetary Sciences, Brown University（布朗大学地球、环境与行星科学系）；Data Science Institute, Brown University（布朗大学数据科学研究院）；Department of Computer Science, Brown University（布朗大学计算机科学系）
E-QRGMM: Efficient Generative Metamodeling for Covariate-Dependent Uncertainty Quantification
School of Data Science（数据科学学院）；School of Management（管理学院）；Department of Decision Analytics and Operations, College of Business, City University of Hong Kong（商业学院决策分析与运营系，香港城市大学）；The Laboratory for AI-Powered Financial Technologies（人工智能赋能金融技术实验室）
Uncertainty quantification for improving radiomic-based models in radiation pneumonitis prediction
Chakri Naruebodindra Medical Institute, Faculty of Medicine, Ramathibodi Hospital, Mahidol University（玛希隆大学拉玛提博迪医院医学院查克里·纳鲁厄博丁德拉医学研究所）；Department of Clinical Epidemiology and Biostatistics, Faculty of Medicine, Ramathibodi Hospital, Mahidol University（玛希隆大学拉玛提博迪医院医学院临床流行病学与生物统计学系）；Department of Diagnostic and Therapeutic Radiology, Faculty of Medicine, Ramathibodi Hospital, Mahidol University（玛希隆大学拉玛提博迪医院医学院诊断与治疗放射学系）
University of Wuppertal（伍珀塔尔大学）；School of Mathematics and Natural Sciences（数学与自然科学学院）
In object detection with deep neural networks, the box-wise objectness score tends to be overconfident, sometimes even indicating high confidence in presence of inaccurate predictions. Hence, the reliability of the prediction and therefore reliable uncertainties are of highest interest. In this work, we present a post processing method that for any given neural network provides predictive uncertainty estimates and quality estimates. These estimates are learned by a post processing model that receives as input a hand-crafted set of transparent metrics in form of a structured dataset. Therefrom,…
Harvard University（哈佛大学）；IBM T.J. Watson Research Center（IBM托马斯·J·沃森研究中心）
Bayesian Neural Networks (BNNs) place priors over the parameters in a neural network. Inference in BNNs, however, is difficult; all inference methods for BNNs are approximate. In this work, we empirically compare the quality of predictive uncertainty estimates for 10 common inference methods on both regression and classification tasks. Our experiments demonstrate that commonly used metrics (e.g. test log-likelihood) can be misleading. Our experiments also indicate that inference innovations designed to capture structure in the posterior do not necessarily produce high quality posterior approxi…
