When deploying machine learning models in high-stakes real-world environments such as health care, it is crucial to accurately assess the uncertainty concerning a model's prediction on abnormal inputs. However, there is a scarcity of literature analyzing this problem on medical data, especially on mixed-type tabular data such as Electronic Health Records. We close this gap by presenting a series of tests including a large variety of contemporary uncertainty estimation techniques, in order to determine whether they are able to identify out-of-distribution (OOD) patients. In contrast to previous work, we design tests on realistic and clinically relevant OOD groups, and run experiments on real-world medical data. We find that almost all techniques fail to achieve convincing results, partly disagreeing with earlier findings.
Unmasking the Chameleons: A Benchmark for Out-of-Distribution Detection in Medical Tabular Data
Amsterdam Public Health Research Institute（阿姆斯特丹公共卫生研究院）；Amsterdam UMC（阿姆斯特丹大学医学中心）；University of Amsterdam（阿姆斯特丹大学）；Institute of Logic, Language and Computation, University of Amsterdam（阿姆斯特丹大学逻辑、语言与计算研究所）；Pacmed（Pacmed公司）
Empirical Evaluation of Out-Of-Distribution Performance of Tabular Foundation Models
Colegio de Ciencias e Ingenierías, Universidad San Francisco de Quito (USFQ)（基多圣弗朗西斯科大学科学与工程学院）；Rey Juan Carlos University（胡安·卡洛斯国王大学）；Facultad de Ingeniería, Universidad Latina de Panamá（巴拿马拉丁大学工程学院）
Center for Risk and Reliability, University of Maryland, College Park（马里兰大学帕克分校风险与可靠性中心）；Department of Civil and Environmental Engineering, University of California, Los Angeles（加利福尼亚大学洛杉矶分校土木与环境工程系）；Garrick Institute for the Risk Sciences, University of California, Los Angeles（加利福尼亚大学洛杉矶分校加里克风险科学研究所）；Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University（香港理工大学工业与系统工程系）；Centre for Advances in Reliability and Safety Limited (CAiRS)（可靠性及安全进展中心有限公司）
There has been a growing interest in deep learning-based prognostic and health management (PHM) for building end-to-end maintenance decision support systems, especially due to the rapid development of autonomous systems. However, the low trustworthiness of PHM hinders its applications in safety-critical assets when handling data from an unknown distribution that differs from the training dataset, referred to as the out-of-distribution (OOD) dataset. To bridge this gap, we propose an uncertainty-informed framework to diagnose faults and meanwhile detect the OOD dataset, enabling the capability …
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Mitigating Overconfidence in Out-of-Distribution Detection by Capturing Extreme Activations
Amsterdam Public Health Research Institute（阿姆斯特丹公共卫生研究院）；Amsterdam UMC（阿姆斯特丹大学医学中心）；University of Amsterdam（阿姆斯特丹大学）；Institute of Logic, Language and Computation（逻辑、语言与计算研究所）；Pacmed（帕克梅德公司）
Reliable OOD Virtual Screening with Extrapolatory Pseudo-Label Matching
Department of Computer Science, University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校计算机科学系）；Eshelman School of Pharmacy, University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校埃舍尔曼药学院）
Discriminatively trained neural classifiers can be trusted, only when the input data comes from the training distribution (in-distribution). Therefore, detecting out-of-distribution (OOD) samples is very important to avoid classification errors. In the context of OOD detection for image classification, one of the recent approaches proposes training a classifier called "confident-classifier" by minimizing the standard cross-entropy loss on in-distribution samples and minimizing the KL divergence between the predictive distribution of OOD samples in the low-density regions of in-distribution and…
University of Pennsylvania（宾夕法尼亚大学）；SRI International（SRI国际）
Deep neural networks (DNNs) are known to produce incorrect predictions with very high confidence on out-of-distribution inputs (OODs). This limitation is one of the key challenges in the adoption of DNNs in high-assurance systems such as autonomous driving, air traffic management, and medical diagnosis. This challenge has received significant attention recently, and several techniques have been developed to detect inputs where the model's prediction cannot be trusted. These techniques detect OODs as datapoints with either high epistemic uncertainty or high aleatoric uncertainty. We demonstrate…
University of Pennsylvania（宾夕法尼亚大学）；SRI International（SRI国际）
Deep neural networks (DNNs) are known to produce incorrect predictions with very high confidence on out-of-distribution (OOD) inputs. This limitation is one of the key challenges in the adoption of deep learning models in high-assurance systems such as autonomous driving, air traffic management, and medical diagnosis. This challenge has received significant attention recently, and several techniques have been developed to detect inputs where the model's prediction cannot be trusted. These techniques use different statistical, geometric, or topological signatures. This paper presents a taxonomy…
It is an important problem in trustworthy machine learning to recognize out-of-distribution (OOD) inputs which are inputs unrelated to the in-distribution task. Many out-of-distribution detection methods have been suggested in recent years. The goal of this paper is to recognize common objectives as well as to identify the implicit scoring functions of different OOD detection methods. We focus on the sub-class of methods that use surrogate OOD data during training in order to learn an OOD detection score that generalizes to new unseen out-distributions at test time. We show that binary discrim…
Duke University（杜克大学）；Air Force Research Laboratory（美国空军研究实验室）
Machine learning methods must be trusted to make appropriate decisions in real-world environments, even when faced with out-of-distribution (OOD) samples. Many current approaches simply aim to detect OOD examples and alert the user when an unrecognized input is given. However, when the OOD sample significantly overlaps with the training data, a binary anomaly detection is not interpretable or explainable, and provides little information to the user. We propose a new model for OOD detection that makes predictions at varying levels of granularity as the inputs become more ambiguous, the model pr…
Negative Label Guided OOD Detection with Pretrained Vision-Language Models
Southern University of Science and Technology（南方科技大学）；Hong Kong Baptist University（香港浸会大学）；University of Melbourne（墨尔本大学）；University of Technology Sydney（悉尼科技大学）；Huazhong Agricultural University（华中农业大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；University of Sydney（悉尼大学）
When Confidence Lacks Concepts: Interpretable OOD Detection via Representation Perturbations
NepAl Applied Mathematics and Informatics Institute for research（尼泊尔应用数学与信息学研究所）；West Virginia University（西弗吉尼亚大学）；Kathmandu University（加德满都大学）；University College London（伦敦大学学院）；University of Aberdeen（阿伯丁大学）
COOkeD: Ensemble-based OOD detection in the era of zero-shot CLIP
Aalborg University（奥尔堡大学）；ENSTA, Institut Polytechnique de Paris（法国国立高等先进技术学校，巴黎综合理工学院）；University of Barcelona（巴塞罗那大学）；Computer Vision Center（计算机视觉中心）
Continual Contrastive Learning on Tabular Data with Out of Distribution
Indonesia Endowment Fund for Education Agency (LPDP)（印度尼西亚教育基金机构（LPDP））；School of Electrical Engineering and Computer Science（电气工程与计算机科学学院）；The University of Queensland（昆士兰大学）；Department of Computing（计算机系）；The Hong Kong Polytechnic University（香港理工大学）
Don't Break the Boundary: Continual Unlearning for OOD Detection Based on Free Energy Repulsion
School of Computer and Electronic Information, Nanjing Normal University（计算机与电子信息学院，南京师范大学）；School of Artificial Intelligence and Information Technology, Nanjing University of Chinese Medicine（人工智能与信息科技学院，南京中医药大学）
Pohang University of Science and Technology (POSTECH)（浦项科技大学）
The capability of reliably detecting out-of-distribution samples is one of the key factors in deploying a good classifier, as the test distribution always does not match with the training distribution in most real-world applications. In this work, we present a deep multi-class data description, termed as Deep-MCDD, which is effective to detect out-of-distribution (OOD) samples as well as classify in-distribution (ID) samples. Unlike the softmax classifier that only focuses on the linear decision boundary partitioning its latent space into multiple regions, our Deep-MCDD aims to find a spherica…
When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Need
Sydney AI Centre, The University of Sydney（悉尼人工智能中心，悉尼大学）；School of Computer Science, Wuhan University（武汉大学计算机学院）；Department of Computer Science, Hong Kong Baptist University（香港 Baptist 大学计算机系）
Darmstadt University of Technology（达姆施塔特工业大学）；Uniklinik Frankfurt（法兰克福大学医院）
Automatic segmentation of ground glass opacities and consolidations in chest computer tomography (CT) scans can potentially ease the burden of radiologists during times of high resource utilisation. However, deep learning models are not trusted in the clinical routine due to failing silently on out-of-distribution (OOD) data. We propose a lightweight OOD detection method that leverages the Mahalanobis distance in the feature space and seamlessly integrates into state-of-the-art segmentation pipelines. The simple approach can even augment pre-trained models with clinically relevant uncertainty …
TECNALIA；University of Deusto（德乌斯托大学）；University of the Basque Country (UPV/EHU)（巴斯克大学（UPV/EHU））；Auckland University of Technology（奥克兰理工大学）；Ulster University（阿尔斯特大学）；Bulgarian Academy of Sciences（保加利亚科学院）
Research around Spiking Neural Networks has ignited during the last years due to their advantages when compared to traditional neural networks, including their efficient processing and inherent ability to model complex temporal dynamics. Despite these differences, Spiking Neural Networks face similar issues than other neural computation counterparts when deployed in real-world settings. This work addresses one of the practical circumstances that can hinder the trustworthiness of this family of models: the possibility of querying a trained model with samples far from the distribution of its tra…
Redesigning Out-of-Distribution Detection on 3D Medical Images
Artificial Intelligence Research Institute (AIRI)（人工智能研究院（AIRI））；Moscow Institute of Physics and Technology（莫斯科物理技术学院）；Skolkovo Institute of Science and Technology（斯科尔科沃科技学院）
针对 3D 医学图像 OOD 检测缺乏临床相关定义的问题，提出以下游模型性能为伪度量定义异常样本，并用 EPD 指标按临床影响评估方法。
Aircraft industry is constantly striving for more efficient design optimization methods in terms of human efforts, computation time, and resource consumption. Hybrid surrogate optimization maintains high results quality while providing rapid design assessments when both the surrogate model and the switch mechanism for eventually transitioning to the HF model are calibrated properly. Feedforward neural networks (FNNs) can capture highly nonlinear input-output mappings, yielding efficient surrogates for aircraft performance factors. However, FNNs often fail to generalize over the out-of-distribu…
We propose a Model Predictive Control (MPC) method for collision-free navigation that uses amortized variational inference to approximate the distribution of optimal control sequences by training a normalizing flow conditioned on the start, goal and environment. This representation allows us to learn a distribution that accounts for both the dynamics of the robot and complex obstacle geometries. We can then sample from this distribution to produce control sequences which are likely to be both goal-directed and collision-free as part of our proposed FlowMPPI sampling-based MPC method. However, …
Classification on long-tailed distributed data is a challenging problem, which suffers from serious class-imbalance and accordingly unpromising performance especially on tail classes. Recently, the ensembling based methods achieve the state-of-the-art performance and show great potential. However, there are two limitations for current methods. First, their predictions are not trustworthy for failure-sensitive applications. This is especially harmful for the tail classes where the wrong predictions is basically frequent. Second, they assign unified numbers of experts to all samples, which is re…
