Conformal Prediction (CP) serves as a robust framework that quantifies uncertainty in predictions made by Machine Learning (ML) models. Unlike traditional point predictors, CP generates statistically valid prediction regions, also known as prediction intervals, based on the assumption of data exchangeability. Typically, the construction of conformal predictions hinges on p-values. This paper, however, ventures down an alternative path, harnessing the power of e-test statistics to augment the efficacy of conformal predictions by introducing a BB-predictor (bounded from the below predictor).
Enhancing Adversarial Robustness with Conformal Prediction: A Framework for Guaranteed Model Reliability
Huaiyin Institute of Technology, Huai'an, China（淮阴职业技术学院）；Chengdu Research Institute, City University of Hong Kong, Chengdu, China（香港城市大学成都研究院）；Saarland University, Saarbrücken, Germany（萨尔兰大学）；Alpha Benito Research, Los Angeles, USA（Alpha Benito研究公司）；City University of Hong Kong, Hong Kong, China（香港城市大学）
Multi-Agent Conformal Prediction with Personalized Statistical Validity
Department of Electronic Systems（电子系统系）；Aalborg University（奥尔堡大学）；Department of Mathematical Sciences（数学科学系）；Institut de Mathématiques de Toulouse（图卢兹数学研究所）；Université de Toulouse（图卢兹大学）
In regression, conformal prediction is a general methodology to construct prediction intervals in a distribution-free manner. Although conformal prediction guarantees strong statistical property for predictive inference, its inherent computational challenge has attracted the attention of researchers in the community. In this paper, we propose a new framework, called Trimmed Conformal Prediction (TCP), based on two stage procedure, a trimming step and a prediction step. The idea is to use a preliminary trimming step to substantially reduce the range of possible values for the prediction interva…
Conformal prediction aims to determine precise levels of confidence in predictions for new objects using past experience. However, the commonly used exchangeable assumptions between the training data and testing data limit its usage in dealing with contaminated testing sets. In this paper, we develop a novel flow-based conformal inference (FCI) method to build predictive sets and infer outliers for complex and high-dimensional data. We leverage ideas from adversarial flow to transfer the input data to a random vector with known distributions. Our roundtrip transformation can map the input data…
Improving the statistical efficiency of cross-conformal prediction
Department of Statistical Sciences, University of Padova（帕多瓦大学统计科学系）；Department of Statistics and Data Science, Carnegie Mellon University（卡内基梅隆大学统计与数据科学系）
An important problem in network analysis is predicting a node attribute using both network covariates, such as graph embedding coordinates or local subgraph counts, and conventional node covariates, such as demographic characteristics. While standard regression methods that make use of both types of covariates may be used for prediction, statistical inference is complicated by the fact that the nodal summary statistics are often dependent in complex ways. We show that under a mild joint exchangeability assumption, a network analog of conformal prediction achieves finite sample validity for a w…
We consider the problem of testing the equality of conditional distributions of a response variable given a vector of covariates between two populations. Such a hypothesis testing problem can be motivated from various machine learning and statistical inference scenarios, including transfer learning and causal predictive inference. We develop a nonparametric test procedure inspired from the conformal prediction framework. The construction of our test statistic combines recent developments in conformal prediction with a novel choice of conformity score, resulting in a weighted rank-sum test stat…
Conformal prediction has been a very popular method of distribution-free predictive inference in recent years in machine learning and statistics. Its popularity stems from the fact that it works as a wrapper around any prediction algorithm such as neural networks or random forests. Exchangeability is at the core of the validity of conformal prediction. The concept of exchangeability is also at the core of rank tests widely known in nonparametric statistics. In this paper, we review the concept of exchangeability and discuss the implications for conformal prediction and rank tests. We provide a…
Conformal prediction methods construct prediction regions for iid data that are valid in finite samples. We provide two parametric conformal prediction regions that are applicable for a wide class of continuous statistical models. This class of statistical models includes generalized linear models (GLMs) with continuous outcomes. Our parametric conformal prediction regions possesses finite sample validity, even when the model is misspecified, and are asymptotically of minimal length when the model is correctly specified. The first parametric conformal prediction region is constructed through b…
Risk assessments to help inform criminal justice decisions have been used in the United States since the 1920s. Over the past several years, statistical learning risk algorithms have been introduced amid much controversy about fairness, transparency and accuracy. In this paper, we focus on accuracy for a large department of probation and parole that is considering a major revision of its current, statistical learning risk methods. Because the content of each offender's supervision is substantially shaped by a forecast of subsequent conduct, forecasts have real consequences. Here we consider th…
Royal Holloway University of London（伦敦大学皇家霍洛威学院）；Centrica plc（森特理克公司）；Politecnico di Milano（米兰理工大学）
In this work we present a clustering technique called \textit{multi-level conformal clustering (MLCC)}. The technique is hierarchical in nature because it can be performed at multiple significance levels which yields greater insight into the data than performing it at just one level. We describe the theoretical underpinnings of MLCC, compare and contrast it with the hierarchical clustering algorithm, and then apply it to real world datasets to assess its performance. There are several advantages to using MLCC over more classical clustering techniques: Once a significance level has been set, ML…
Enhancing Electricity-System Resilience with Adaptive Robust Optimization and Conformal Uncertainty Characterization
Heinz College of Information Systems and Public Policy, Carnegie Mellon University（信息系统与公共政策学院，卡内基梅隆大学）；Carnegie Mellon Electricity Industry Center（卡内基梅隆电力产业中心）；Wilton E. Scott Institute for Energy Innovation（威利特·E·斯科特能源创新研究所）；Department of Engineering and Public Policy, Carnegie Mellon Electricity Industry Center（工程与公共政策系，卡内基梅隆电力产业中心）；Department of Electrical and Computer Engineering, Heinz College of Information Systems and Public Policy（电气与计算机工程系，信息系统与公共政策学院）；Department of Integrated Systems Engineering, The Ohio State University（整合系统工程系，俄亥俄州立大学）
Statistical Guarantees in Synthetic Data through Conformal Adversarial Generation
WorkOnward Inc WOW AI Labs California（WorkOnward公司WOW AI实验室加州）；Department of Computer Science California State University Long Beach（计算机科学系加州州立大学长滩分校）；Department of Computer Science Emory University（计算机科学系埃默里大学）
Enhancing the conformal predictability of context-aware recommendation systems by using Deep Autoencoders
Faculty of Science of Tunis, University of Tunis El-Manar（突尼斯大学理工学院）；SVKM’s Dwarkadas J. Sanghvi College of Engineering（SVKM德瓦尔卡斯贾亚甘吉学院）；The Maersk Mc-Kinney Moller Institute, University of Southern Denmark（马士基麦金尼莫勒研究所，南部丹麦大学）；Department of Software Science, Tallinn University of Technology（塔林技术大学软件科学系）
This note proposes a procedure for enhancing the quality of probabilistic prediction algorithms via betting against their predictions. It is inspired by the success of the conformal test martingales that have been developed recently.
Adversarial training has been shown to be one of the most effective approaches to improve the robustness of deep neural networks. It is formalized as a min-max optimization over model weights and adversarial perturbations, where the weights can be optimized through gradient descent methods like SGD. In this paper, we show that treating model weights as random variables allows for enhancing adversarial training through \textbf{S}econd-Order \textbf{S}tatistics \textbf{O}ptimization (S$^2$O) with respect to the weights. By relaxing a common (but unrealistic) assumption of previous PAC-Bayesian f…
Reliable Statistical Guarantees for Conformal Predictors with Small Datasets
ETSIAE-UPM - School of Aeronautics, Universidad Politécnica de Madrid（西班牙马德里理工大学航空航天学院）；Institute for Cross-Disciplinary Physics and Complex Systems (IFISC), CSIC-UIB（交叉学科物理与复杂系统研究所（IFISC），CSIC-UIB）；Center for Computational Simulation, Universidad Politécnica de Madrid, Campus de Montegancedo, Boadilla del Monte（计算模拟中心，马德里理工大学蒙特加森多校区，博阿迪利亚德尔蒙特）
Non-Exchangeable Conformal Language Generation with Nearest Neighbors
IT University of Copenhagen（哥本哈根信息技术大学）；Pioneer Centre for Artificial Intelligence（先锋人工智能中心）；Instituto de Telecomunicações（电信研究所）；Instituto Superior Técnico, Universidade de Lisboa（里斯本大学高等技术学院）；Unbabel（昂巴贝尔公司）
Enhancing Federated Class-Incremental Learning via Spatial-Temporal Statistics Aggregation
University of Chinese Academy of Sciences（中国科学院大学）；Tianjin University（天津大学）；University of Science and Technology of China（中国科学技术大学）；University of Rochester（罗切斯特大学）；State Key Laboratory of Cyberspace Security Defense, Institute of Information Engineering（信息工程研究所网络空间安全防御国家重点实验室）
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Carle’s Illinois College of Medicine（卡尔伊利诺伊医学院）；GE Healthcare（GE医疗）；University of Chicago（芝加哥大学）
S2O: Enhancing Adversarial Training with Second-Order Statistics of Weights
Department of Computer Science, University of Exeter（埃克塞特大学计算机科学系）；National Mobile Communications Research Laboratory, Southeast University（东南大学国家移动通信研究实验室）；Department of Computer Science, University of Liverpool（利物浦大学计算机科学系）
