For safety, medical AI systems undergo thorough evaluations before deployment, validating their predictions against a ground truth which is assumed to be fixed and certain. However, this ground truth is often curated in the form of differential diagnoses. While a single differential diagnosis reflects the uncertainty in one expert assessment, multiple experts introduce another layer of uncertainty through disagreement. Both forms of uncertainty are ignored in standard evaluation which aggregates these differential diagnoses to a single label. In this paper, we show that ignoring uncertainty leads to overly optimistic estimates of model performance, therefore underestimating risk associated with particular diagnostic decisions. To this end, we propose a statistical aggregation approach, where we infer a distribution on probabilities of underlying medical condition candidates themselves, based on observed annotations. This formulation naturally accounts for the potential disagreements between different experts, as well as uncertainty stemming from individual differential diagnoses, capturing the entire ground truth uncertainty. Our approach boils down to generating multiple samples of medical condition probabilities, then evaluating and averaging performance metrics based on these sampled probabilities. In skin condition classification, we find that a large portion of the dataset exhibits significant ground truth uncertainty and standard evaluation severely over-estimates performance without providing uncertainty estimates. In contrast, our framework provides uncertainty estimates on common metrics of interest such as top-k accuracy and average overlap, showing that performance can change multiple percentage points. We conclude that, while assuming a crisp ground truth can be acceptable for many AI applications, a more nuanced evaluation protocol should be utilized in medical diagnosis.
A Ground-Truth Framework for Uncertainty Disentanglement with Posterior Risk
University of Tübingen（蒂宾根大学）；Hertie Institute for AI in Brain Health（赫蒂脑健康人工智能研究所）；Tübingen AI Center（蒂宾根人工智能中心）；Charité–Universitätsmedizin Berlin（柏林夏里特医学院）；Bernstein Center for Computational Neuroscience Berlin（柏林伯恩斯坦计算神经科学中心）
From Propagation to Prediction: Point-level Uncertainty Evaluation of MLS Point Clouds under Limited Ground Truth
Chair of Engineering Geodesy, TUM School of Engineering and Design, Technical University of Munich（工程测绘学系，技术大学慕尼黑工程与设计学院）；TUM Leonhard Obermeyer Center, Technical University of Munich（技术大学慕尼黑Leonhard Obermeyer中心）；CV4DT, University of Cambridge（剑桥大学CV4DT）
University at Buffalo（布法罗大学）；Penn State Erie（宾夕法尼亚州立大学伊利分校）
Deep Neural Networks (DNNs), despite their tremendous success in recent years, could still cast doubts on their predictions due to the intrinsic uncertainty associated with their learning process. Ensemble techniques and post-hoc calibrations are two types of approaches that have individually shown promise in improving the uncertainty calibration of DNNs. However, the synergistic effect of the two types of methods has not been well explored. In this paper, we propose a truth discovery framework to integrate ensemble-based and post-hoc calibration methods. Using the geometric variance of the en…
A personalized Uncertainty Quantification framework for patient survival models: estimating individual uncertainty of patients with metastatic brain tumors in the absence of ground truth
UCL（伦敦大学学院）；Meta AI；UC Berkeley（加州大学伯克利分校）；University of Oxford（牛津大学）；MILA（蒙特利尔学习算法研究所）；Inflection AI（Inflection人工智能）；Cohere；FLAIR（未来人工智能研究中心）
Adaptive curricula in reinforcement learning (RL) have proven effective for producing policies robust to discrepancies between the train and test environment. Recently, the Unsupervised Environment Design (UED) framework generalized RL curricula to generating sequences of entire environments, leading to new methods with robust minimax regret properties. Problematically, in partially-observable or stochastic settings, optimal policies may depend on the ground-truth distribution over aleatoric parameters of the environment in the intended deployment setting, while curriculum learning necessarily…
Between the Layers Lies the Truth: Uncertainty Estimation in LLMs Using Intra-Layer Local Information Scores
Faculty of Data and Decision Sciences, Technion --- Israel Institute of Technology（数据与决策科学学院，技术离子研究所）；Faculty of Computer Science, Technion --- Israel Institute of Technology（计算机科学学院，技术离子研究所）；Faculty of Biomedical Engineering, Technion --- Israel Institute of Technology（生物医学工程学院，技术离子研究所）
Google Research（谷歌研究院）；Stanford University（斯坦福大学）；UC Berkeley（加州大学伯克利分校）
Learning under one-sided feedback (i.e., where we only observe the labels for examples we predicted positively on) is a fundamental problem in machine learning -- applications include lending and recommendation systems. Despite this, there has been surprisingly little progress made in ways to mitigate the effects of the sampling bias that arises. We focus on generalized linear models and show that without adjusting for this sampling bias, the model may converge suboptimally or even fail to converge to the optimal solution. We propose an adaptive approach that comes with theoretical guarantees …
We study the problem of reinforcement learning for a task encoded by a reward machine. The task is defined over a set of properties in the environment, called atomic propositions, and represented by Boolean variables. One unrealistic assumption commonly used in the literature is that the truth values of these propositions are accurately known. In real situations, however, these truth values are uncertain since they come from sensors that suffer from imperfections. At the same time, reward machines can be difficult to model explicitly, especially when they encode complicated tasks. We develop a…
Breaking the Chains of Probability: Neutrosophic Logic as a New Framework for Epistemic Uncertainty in Large Language Models
Universidad Bolivariana del Ecuador, Coordinación Académica de Posgrado（巴尔干大学厄瓜多尔分校，研究生院）；Universidad de Guayaquil（瓜亚基尔大学）；Universidad Bernardo O’Higgins（伯纳多·奥希金斯大学）；Mathematics, Physics, and Natural Sciences Division, University of New Mexico（新墨西哥大学数学、物理和自然科学系）
Improving Generalization and Uncertainty Quantification of Photometric Redshift Models
Department of Physics and Astronomy, UCLA, Los Angeles, CA 90095, USA（物理与天文学系，UCLA）；Computer Science Department, Southern Oregon University, Ashland, OR 97520, USA（计算机科学系，索纳德奥尔大学）；Department of Physics, University of Richmond, Richmond, VA 23173, USA（物理系，里奇蒙大学）
Reliable uncertainty quantification in deep neural networks is very crucial in safety-critical applications such as automated driving for trustworthy and informed decision-making. Assessing the quality of uncertainty estimates is challenging as ground truth for uncertainty estimates is not available. Ideally, in a well-calibrated model, uncertainty estimates should perfectly correlate with model error. We propose a novel error aligned uncertainty optimization method and introduce a trainable loss function to guide the models to yield good quality uncertainty estimates aligning with the model e…
Fraunhofer IAIS（弗劳恩霍夫智能分析与信息系统研究所）；Fraunhofer Center for Machine Learning（弗劳恩霍夫机器学习中心）；IAV GmbH（IAV有限公司）；Volkswagen Group Innovation（大众汽车集团创新部）
Statistical models are inherently uncertain. Quantifying or at least upper-bounding their uncertainties is vital for safety-critical systems such as autonomous vehicles. While standard neural networks do not report this information, several approaches exist to integrate uncertainty estimates into them. Assessing the quality of these uncertainty estimates is not straightforward, as no direct ground truth labels are available. Instead, implicit statistical assessments are required. For regression, we propose to evaluate uncertainty realism -- a strict quality criterion -- with a Mahalanobis dist…
In patients with atrial fibrillation, local activation time (LAT) maps are routinely used for characterising patient pathophysiology. The gradient of LAT maps can be used to calculate conduction velocity (CV), which directly relates to material conductivity and may provide an important measure of atrial substrate properties. Including uncertainty in CV calculations would help with interpreting the reliability of these measurements. Here, we build upon a recent insight into reduced-rank Gaussian processes (GP) to perform probabilistic interpolation of uncertain LAT directly on human atrial mani…
Obtaining reliable and accurate quantification of uncertainty estimates from deep neural networks is important in safety-critical applications. A well-calibrated model should be accurate when it is certain about its prediction and indicate high uncertainty when it is likely to be inaccurate. Uncertainty calibration is a challenging problem as there is no ground truth available for uncertainty estimates. We propose an optimization method that leverages the relationship between accuracy and uncertainty as an anchor for uncertainty calibration. We introduce a differentiable accuracy versus uncert…
University of California, Los Angeles（加州大学洛杉矶分校）；University of Miami（迈阿密大学）
Contextual bandit algorithms -- a class of multi-armed bandit algorithms that exploit the contextual information -- have been shown to be effective in solving sequential decision making problems under uncertainty. A common assumption adopted in the literature is that the realized (ground truth) reward by taking the selected action is observed by the learner at no cost, which, however, is not realistic in many practical scenarios. When observing the ground truth reward is costly, a key challenge for the learner is how to judiciously acquire the ground truth by assessing the benefits and costs i…
Beyond Accuracy: Evaluating Posterior Fidelity of Diffusion Inverse Solvers
Department of Statistics, University of Michigan（密歇根大学统计学系）；Department of EECS, University of Michigan（密歇根大学电子工程与计算机科学系）；Department of Statistics, Rutgers University（罗格斯大学统计学系）
