Deep Neural Networks (DNNs) achieve state-of-the-art performance on numerous applications. However, it is difficult to tell beforehand if a DNN receiving an input will deliver the correct output since their decision criteria are usually nontransparent. A DNN delivers the correct output if the input is within the area enclosed by its generalization envelope. In this case, the information contained in the input sample is processed reasonably by the network. It is of large practical importance to assess at inference time if a DNN generalizes correctly. Currently, the approaches to achieve this goal are investigated in different problem set-ups rather independently from one another, leading to three main research and literature fields: predictive uncertainty, out-of-distribution detection and adversarial example detection. This survey connects the three fields within the larger framework of investigating the generalization performance of machine learning methods and in particular DNNs. We underline the common ground, point at the most promising approaches and give a structured overview of the methods that provide at inference time means to establish if the current input is within the generalization envelope of a DNN.
Language Technologies Institute, Carnegie Mellon University（卡内基梅隆大学语言技术研究所）；Computer Science Department, Stanford University（斯坦福大学计算机科学系）
With increasing deployment of machine learning systems in various real-world tasks, there is a greater need for accurate quantification of predictive uncertainty. While the common goal in uncertainty quantification (UQ) in machine learning is to approximate the true distribution of the target data, many works in UQ tend to be disjoint in the evaluation metrics utilized, and disparate implementations for each metric lead to numerical results that are not directly comparable across different works. To address this, we introduce Uncertainty Toolbox, an open-source python library that helps to ass…
Modern predictive analytics underpinned by machine learning techniques has become a key enabler to the automation of data-driven decision making. In the context of business process management, predictive analytics has been applied to making predictions about the future state of an ongoing business process instance, for example, when will the process instance complete and what will be the outcome upon completion. Machine learning models can be trained on event log data recording historical process execution to build the underlying predictive models. Multiple techniques have been proposed so far…
Predictive process analytics focuses on predicting the future states of running instances of a business process. While advanced machine learning techniques have been used to increase accuracy of predictions, the resulting predictive models lack transparency. Current explainable machine learning methods, such as LIME and SHAP, can be used to interpret black box models. However, it is unclear how fit for purpose these methods are in explaining process predictive models. In this paper, we draw on evaluation measures used in the field of explainable AI and propose functionally-grounded evaluation …
Predictive algorithms have a powerful potential to offer benefits in areas as varied as medicine or education. However, these algorithms and the data they use are built by humans, consequently, they can inherit the bias and prejudices present in humans. The outcomes can systematically repeat errors that create unfair results, which can even lead to situations of discrimination (e.g. gender, social or racial). In order to illustrate how important is to count with a diverse training dataset to avoid bias, we manipulate a well-known facial expression recognition dataset to explore gender bias and…
Robust Predictive Uncertainty and Double Descent in Contaminated Bayesian Random Features
The University of Manchester, UK（英国曼彻斯特大学）；University of Pisa, Italy（意大利比萨大学）；Nanyang Technological University, Singapore（新加坡南洋理工大学）；Max Planck Institute for Mathematics in the Sciences, Germany（德国马克斯·普朗克数学研究所）
A Conformal Predictive Measure for Assessing Catastrophic Forgetting
Communication Systems Department, EURECOM, France（EURECOM通信系统部）；Andalusian Institute of Data Science and Computational Intelligence (DaSCI)（安达卢西亚数据科学与计算智能研究所）；Department of Computer Science and Artificial Intelligence, University of Granada, Spain（格拉纳达大学计算机科学与人工智能系）
Assessing the Generalizability of a Performance Predictive Model
Jožef Stefan Institute（约瑟夫·斯特凡研究所）；Jožef Stefan International Postgraduate School（约瑟夫·斯特凡国际研究生院）；Leiden University（莱顿大学）；Stellenbosch University（斯泰伦博斯大学）；Gulf University for Science and Technology（海湾科技大学）；Sorbonne Université（索邦大学）；CNRS（法国国家科学研究中心）
College of Information and Computer Sciences, University of Massachusetts, Amherst（马萨诸塞大学阿默斯特分校信息与计算机科学学院）
Bayesian Dark Knowledge is a method for compressing the posterior predictive distribution of a neural network model into a more compact form. Specifically, the method attempts to compress a Monte Carlo approximation to the parameter posterior into a single network representing the posterior predictive distribution. Further, the authors show that this approach is successful in the classification setting using a student network whose architecture matches that of a single network in the teacher ensemble. In this work, we examine the robustness of Bayesian Dark Knowledge to higher levels of poster…
University of Chicago（芝加哥大学）；Toyota Technological Institute at Chicago（芝加哥丰田工业大学）
In this paper, we study a bi-criterion framework for assessing scoring functions in the context of binary classification. The positive and negative predictive values (ppv and npv, respectively) are conditional probabilities of the true label matching a classifier's predicted label. The usual classification error rate is a linear combination of these probabilities, and therefore, concentration inequalities for the error rate do not yield confidence intervals for the two separate predictive values. We study generalization properties of scoring functions with respect to predictive values by deriv…
Most work on supervised learning research has focused on marginal predictions. In decision problems, joint predictive distributions are essential for good performance. Previous work has developed methods for assessing low-order predictive distributions with inputs sampled i.i.d. from the testing distribution. With low-dimensional inputs, these methods distinguish agents that effectively estimate uncertainty from those that do not. We establish that the predictive distribution order required for such differentiation increases greatly with input dimension, rendering these methods impractical. To…
Dipartimento di Informatica Università di Pisa（比萨大学信息学院（Dipartimento di Informatica Università di Pisa））
The paper introduces concentric Echo State Network, an approach to design reservoir topologies that tries to bridge the gap between deterministically constructed simple cycle models and deep reservoir computing approaches. We show how to modularize the reservoir into simple unidirectional and concentric cycles with pairwise bidirectional jump connections between adjacent loops. We provide a preliminary experimental assessment showing how concentric reservoirs yield to superior predictive accuracy and memory capacity with respect to single cycle reservoirs and deep reservoir models.
Generalized Conformal Predictive Systems Under Distributional Shifts
IDLab Seminar for Statistics（统计研究所研讨会）；Department of Electronics（电子系）；ETH Zurich（苏黎世联邦理工学院）；Information Systems Zurich, Switzerland（苏黎世信息系统，瑞士）；Ghent University（根特大学）
Institute of Computer Science（计算机科学研究所）；Campus Institute Data Science（校园数据科学研究所）；University of Goettingen（哥廷根大学）
An important step for limiting the negative impact of natural disasters is rapid damage assessment after a disaster occurred. For instance, building damage detection can be automated by applying computer vision techniques to satellite imagery. Such models operate in a multi-domain setting: every disaster is inherently different (new geolocation, unique circumstances), and models must be robust to a shift in distribution between disaster imagery available for training and the images of the new event. Accordingly, estimating real-world performance requires an out-of-domain (OOD) test set. Howeve…
School of Microelectronics and Communication Engineering, Chongqing University（重庆大学微电子与通信工程学院）
The class imbalance problem is important and challenging. Ensemble approaches are widely used to tackle this problem because of their effectiveness. However, existing ensemble methods are always applied into original samples, while not considering the structure information among original samples. The limitation will prevent the imbalanced learning from being better. Besides, research shows that the structure information among samples includes local and global structure information. Based on the analysis above, an imbalanced ensemble algorithm with the deep sample pre-envelope network (DSEN) an…
Assessing the predicting power of GPS data for aftershocks forecasting
The Institute of Statistical Mathematics, Research Organization of Information and Systems（信息与系统研究机构统计数学研究所）；Université Paris-Saclay（巴黎萨克雷大学）；CNRS（法国国家科学研究中心）；INRIA（法国国家信息与自动化研究所）
Forensic examination of evidence like firearms and toolmarks, traditionally involves a visual and therefore subjective assessment of similarity of two questioned items. Statistical models are used to overcome this subjectivity and allow specification of error rates. These models are generally quite complex and produce abstract results at different levels of the analysis. Presenting such metrics and complicated results to examiners is challenging, as examiners generally do not have substantial statistical training to accurately interpret results. This creates distrust in statistical modelling a…
Humans rely heavily on shapes as a primary cue for object recognition. As secondary cues, colours and textures are also beneficial in this regard. Convolutional neural networks (CNNs), an imitation of biological neural networks, have been shown to exhibit conflicting properties. Some studies indicate that CNNs are biased towards textures whereas, another set of studies suggests shape bias for a classification task. However, they do not discuss the role of colours, implying its possible humble role in the task of object recognition. In this paper, we empirically investigate the importance of co…
The University of Sydney（悉尼大学）；Macquarie University（麦考瑞大学）；Western Sydney University（西悉尼大学）
The increasing prevalence of marine pollution during the past few decades motivated recent research to help ease the situation. Typical water quality assessment requires continuous monitoring of water and sediments at remote locations with labour intensive laboratory tests to determine the degree of pollution. We propose an automated framework where we formalise a predictive model using Machine Learning to infer the water quality and level of pollution using collected water and sediments samples. One commonly encountered difficulty performing statistical analysis with water and sediment is the…
Machine learning models $-$ now commonly developed to screen, diagnose, or predict health conditions $-$ are evaluated with a variety of performance metrics. An important first step in assessing the practical utility of a model is to evaluate its average performance over an entire population of interest. In many settings, it is also critical that the model makes good predictions within predefined subpopulations. For instance, showing that a model is fair or equitable requires evaluating the model's performance in different demographic subgroups. However, subpopulation performance metrics are t…
Open-weight genome language model safeguards: Assessing robustness via adversarial fine-tuning
Center for Health Security（健康安全中心）；Johns Hopkins Bloomberg School of Public Health（约翰霍普金斯大学布隆伯格公共卫生学院）；Department of Chemistry（化学系）；University of Oxford（牛津大学）；Stanford University School of Medicine（斯坦福大学医学院）；Department of Chemistry & Kavli Institute for Nanoscience Discovery（化学系及卡弗里纳米科学发现研究所）
