Vladimir Vovk
This note introduces the method of cross-conformal prediction, which is a hybrid of the methods of inductive conformal prediction and cross-validation, and studies its validity and predictive efficiency empirically.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
Beyond Conformal Predictors: Adaptive Conformal Inference with Confidence Predictors
本研究证明自适应保形推断的覆盖保证适用于更广泛的非保形置信预测器，在非可交换数据下提供计算优势并提升预测效率。
Efficiency criteria for conformal prediction, such as \emph{observed fuzziness} (i.e., the sum of p-values associated with false labels), are commonly used to \emph{evaluate} the performance of given conformal predictors. Here, we investigate whether it is possible to exploit efficiency criteria to \emph{learn} classifiers, both conformal predictors and point classifiers, by using such criteria as training objective functions. The proposed idea is implemented for the problem of binary classification of hand-written digits. By choosing a 1-dimensional model class (with one real-valued free para…
CAOS: Conformal Aggregation of One-Shot Predictors
CAOS通过自适应聚合多个一次性预测器，利用留一校准方案实现有效覆盖，比传统方法在预测集大小上更小且保持可靠覆盖。
Conformalized Credal Set Predictors
本文提出了一种基于保形预测的分类任务Credal集预测方法，能够以高概率保证预测的有效性，并展示了其在自然语言推理任务中的适用性。
Conformal e-prediction
该研究从现代视角系统重审共形预测与前身共形e预测的关系，指出共形e预测具备易设计条件预测器、交叉预测器有效性有保障等独有优势，且可模拟共形预测的部分优势。
Conventional frequentist learning is known to yield poorly calibrated models that fail to reliably quantify the uncertainty of their decisions. Bayesian learning can improve calibration, but formal guarantees apply only under restrictive assumptions about correct model specification. Conformal prediction (CP) offers a general framework for the design of set predictors with calibration guarantees that hold regardless of the underlying data generation mechanism. However, when training data are limited, CP tends to produce large, and hence uninformative, predicted sets. This paper introduces a no…
Amyotrophic Lateral Sclerosis (ALS) is a neurodegenerative disease characterized by a rapid motor decline, leading to respiratory failure and subsequently to death. In this context, researchers have sought for models to automatically predict disease progression to assisted ventilation in ALS patients. However, the clinical translation of such models is limited by the lack of insight 1) on the risk of error for predictions at patient-level, and 2) on the most adequate time to administer the non-invasive ventilation. To address these issues, we combine Conformal Prediction (a machine learning fr…
Cross-Validation Conformal Risk Control
本文提出基于交叉验证的共形风险控制方法 CV-CRC，将折刀极小极大思想从共形预测扩展到风险控制，在有限数据下提供平均风险理论保证并减小集合预测器大小。
Making Conformal Predictors Robust in Healthcare Settings: a Case Study on EEG Classification
本文评估了多种符合预测方法在EEG癫痫分类中的表现，发现个性化校准策略可提升覆盖度20%以上，同时保持预测集大小相近。
Conformal prediction is a method of producing prediction sets that can be applied on top of a wide range of prediction algorithms. The method has a guaranteed coverage probability under the standard IID assumption regardless of whether the assumptions (often considerably more restrictive) of the underlying algorithm are satisfied. However, for the method to be really useful it is desirable that in the case where the assumptions of the underlying algorithm are satisfied, the conformal predictor loses little in efficiency as compared with the underlying algorithm (whereas being a conformal predi…
Conformal predictors, introduced by Vovk et al. (2005), serve to build prediction intervals by exploiting a notion of conformity of the new data point with previously observed data. In the present paper, we propose a novel method for constructing prediction intervals for the response variable in multivariate linear models. The main emphasis is on sparse linear models, where only few of the covariates have significant influence on the response variable even if their number is very large. Our approach is based on combining the principle of conformal prediction with the $\ell_1$ penalized least s…
Modern black-box predictive models are often accompanied by weak performance guarantees that only hold asymptotically in the size of the dataset or require strong parametric assumptions. In response to this, split conformal prediction represents a promising avenue to obtain finite-sample guarantees under minimal distribution-free assumptions. Although prediction set validity most often concerns marginal coverage, we explore the related but different guarantee of tolerance regions, reformulating known results in the language of nested prediction sets and extending on the duality between margina…
Enhancing Conformal Prediction Using E-Test Statistics
本文提出用E检验统计量替代传统p值，并引入BB预测器来增强共形预测的效能。
While the use of deep learning in drug discovery is gaining increasing attention, the lack of methods to compute reliable errors in prediction for Neural Networks prevents their application to guide decision making in domains where identifying unreliable predictions is essential, e.g. precision medicine. Here, we present a framework to compute reliable errors in prediction for Neural Networks using Test-Time Dropout and Conformal Prediction. Specifically, the algorithm consists of training a single Neural Network using dropout, and then applying it N times to both the validation and test sets,…
Weighted Aggregation of Conformity Scores for Classification
本文提出一种结合多个得分函数并最小化预测集大小的共形预测方法，在保持有效覆盖的同时提升了分类任务中预测的效率与实用性。
Universality of conformal prediction under the assumption of randomness
本文研究独立同分布数据下是否存在比保形预测器更高效且同样有效的预测器，证明保形预测器具有普适性，效率提升空间有限，并给出更实用且部分最优的结果。
This paper applies conformal prediction techniques to compute simultaneous prediction bands and clustering trees for functional data. These tools can be used to detect outliers and clusters. Both our prediction bands and clustering trees provide prediction sets for the underlying stochastic process with a guaranteed finite sample behavior, under no distributional assumptions. The prediction sets are also informative in that they correspond to the high density region of the underlying process. While ordinary conformal prediction has high computational cost for functional data, we use the induct…
Generalized Venn and Venn-Abers Calibration with Applications in Conformal Prediction
本文提出统一的Venn和Venn-Abers校准框架，将二元分类扩展至一般损失函数预测问题，生成有限样本下边际校准的集值预测，并提出Venn多校准方法，在保形预测中恢复并扩展了多种特例。
Confident Object Detection via Conformal Prediction and Conformal Risk Control: an Application to Railway Signaling
本文面向铁路信号检测，基于驾驶员视角图像数据集和先进目标检测器，评估多种保形预测方法并提出保形风险控制新方法，以提供形式化保证的不确定性界限。
In this paper, we apply conformal prediction to time series data. Conformal prediction isa method that produces predictive regions given a confidence level. The regions outputs arealways valid under the exchangeability assumption. However, this assumption does not holdfor the time series data because there is a link among past, current, and future observations.Consequently, the challenge of applying conformal predictors to the problem of time seriesdata lies in the fact that observations of a time series are dependent and therefore do notmeet the exchangeability assumption. This paper aims to …
This paper explores conformal prediction in the learning under privileged information (LUPI) paradigm. We use the SVM+ realization of LUPI in an inductive conformal predictor, and apply it to the MNIST benchmark dataset and three datasets in drug discovery. The results show that using privileged information produces valid models and improves efficiency compared to standard SVM, however the improvement varies between the tested datasets and is not substantial in the drug discovery applications. More importantly, using SVM+ in a conformal prediction framework enables valid prediction intervals a…
The main objective of Prognostics and Health Management is to estimate the Remaining Useful Lifetime (RUL), namely, the time that a system or a piece of equipment is still in working order before starting to function incorrectly. In recent years, numerous machine learning algorithms have been proposed for RUL estimation, mainly focusing on providing more accurate RUL predictions. However, there are many sources of uncertainty in the problem, such as inherent randomness of systems failure, lack of knowledge regarding their future states, and inaccuracy of the underlying predictive models, makin…
Conformal Prediction is a machine learning methodology that produces valid prediction regions under mild conditions. In this paper, we explore the application of making predictions over multiple data sources of different sizes without disclosing data between the sources. We propose that each data source applies a transductive conformal predictor independently using the local data, and that the individual predictions are then aggregated to form a combined prediction region. We demonstrate the method on several data sets, and show that the proposed method produces conservatively valid prediction…
On the Expected Size of Conformal Prediction Sets
该研究针对共形预测集规模缺乏有限样本分析的问题，在拆分共形预测框架下理论量化预测集期望规模，推导可经验计算的点估计与区间界，并通过真实数据集的回归和分类实验验证有效性。
Conditional validity of heteroskedastic conformal regression
本文研究异方差回归中保形预测区间的条件有效性问题，通过归一化和Mondrian保形预测等方法构建自适应预测区间，并从理论和实验上系统比较了这些方法的条件有效性及其对数据分布假设的依赖关系。
Randomness, exchangeability, and conformal prediction
本文利用随机性函数理论重述保形预测在 IID 假设下的通用性，证明有效置信预测器可转化为保形预测器且效率损失很小。
Conformal prediction is a powerful distribution-free tool for uncertainty quantification, establishing valid prediction intervals with finite-sample guarantees. To produce valid intervals which are also adaptive to the difficulty of each instance, a common approach is to compute normalized nonconformity scores on a separate calibration set. Self-supervised learning has been effectively utilized in many domains to learn general representations for downstream predictors. However, the use of self-supervision beyond model pretraining and representation learning has been largely unexplored. In this…
Leave-One-Out-, Bootstrap- and Cross-Conformal Anomaly Detectors
为解决异常检测中校准数据不足的问题，基于共形预测提出留一法、自助法和交叉共形方法，在控制第一类错误率的同时提高数据效率。
Most existing examples of full conformal predictive systems, split-conformal predictive systems, and cross-conformal predictive systems impose severe restrictions on the adaptation of predictive distributions to the test object at hand. In this paper we develop split-conformal and cross-conformal predictive systems that are fully adaptive. Our method consists in calibrating existing predictive systems; the input predictive system is not supposed to satisfy any properties of validity, whereas the output predictive system is guaranteed to be calibrated in probability. It is interesting that the …
Decomposition-Based Modular Conformal Prediction for Two-Stage Modeling
提出一种针对两阶段顺序模型的共形预测框架，通过将预测残差分解为阶段特定成分，利用族系错误率控制校准缩放参数，实现不确定性归因和鲁棒覆盖。
The Penalized Inverse Probability Measure for Conformal Classification
本文提出 PIP 非一致性评分及其正则化版本 RePIP，用于联合优化保形分类的集合效率与单元素预测信息量，并在作物和杂草图像分类中验证其优势。
Adaptive Conformal Inference by Particle Filtering under Hidden Markov Models
本文提出一种基于粒子滤波的自适应共形推断框架，通过加权粒子近似隐状态后验分布，在隐马尔可夫模型下实现长期一步及多步推断的聚合覆盖水平，并经实时目标定位仿真验证有效性。
Motivated by the pressing request of methods able to create prediction sets in a general regression framework for a multivariate functional response and pushed by new methodological advancements in non-parametric prediction for functional data, we propose a set of conformal predictors that produce finite-sample either valid or exact multivariate simultaneous prediction bands under the mild assumption of exchangeable regression pairs. The fact that the prediction bands can be built around any regression estimator and that can be easily found in closed form yields a very widely usable method, wh…
Missingness-Aware Conformal Prediction Under Cross-Hospital Distribution Shift
针对跨医院分布偏移下的死亡预测，提出缺失感知共形校准，按测量缺失分组进行蒙德里安校准，减少最差组覆盖差距，并揭示合并评估的局限性。
Improving Expert Predictions with Conformal Prediction
本文提出一种基于保形预测的自动决策支持系统，通过提供预测集而非单个标签并强制专家从中选择，精确权衡误导频率与任务难度，帮助专家提升多类分类准确率且对分类器精度具鲁棒性。
Test-time Recalibration of Conformal Predictors Under Distribution Shift Based on Unlabeled Examples
针对分布偏移导致保形预测器校准失效且新分布缺少标签的问题，提出基于无标签示例预测截断阈值的测试时重新校准方法。
Conformal Unlearning: A New Paradigm for Unlearning in Conformal Predictors
本文提出了一种新的符合性反学习范式，通过有限样本和不确定性的保证，有效移除特定数据点的误覆盖，同时保持预测的准确性。
Most machine learning classifiers give predictions for new examples accurately, yet without indicating how trustworthy predictions are. In the medical domain, this hampers their integration in decision support systems, which could be useful in the clinical practice. We use a supervised learning approach that combines Ensemble learning with Conformal Predictors to predict conversion from Mild Cognitive Impairment to Alzheimer's Disease. Our goal is to enhance the classification performance (Ensemble learning) and complement each prediction with a measure of credibility (Conformal Predictors). O…
Self-Organized Conformal Prediction: Reducing Regional Coverage Gaps with Unsupervised Group Discovery
提出自组织共形预测（SOCP），利用自组织映射（SOM）发现输入空间中的组，并在测试时从最佳匹配单元（BMU）或固定网格邻域提取局部校准缓冲区，以减少区域覆盖缺口，无需监督划分或重新训练预测器。
Conformal prediction methods construct prediction regions for iid data that are valid in finite samples. We provide two parametric conformal prediction regions that are applicable for a wide class of continuous statistical models. This class of statistical models includes generalized linear models (GLMs) with continuous outcomes. Our parametric conformal prediction regions possesses finite sample validity, even when the model is misspecified, and are asymptotically of minimal length when the model is correctly specified. The first parametric conformal prediction region is constructed through b…
Conformal Prediction Assessment: A Framework for Conditional Coverage Evaluation and Selection
本文提出CPA框架，通过训练可靠性估计器评估条件覆盖，引入CVI指数分解可靠性并证明其一致性，实验表明CPA能有效诊断局部失效模式。
CP$^2$: Leveraging Geometry for Conformal Prediction via Canonicalization
针对几何数据偏移破坏共形预测保证的问题，提出将几何位姿规范化信息融入共形过程，以恢复覆盖保证并提升鲁棒性，实验验证其优于等变与增强基线。
It is challenging to deal with censored data, where we only have access to the incomplete information of survival time instead of its exact value. Fortunately, under linear predictor assumption, people can obtain guaranteed coverage for the confidence band of survival time using methods like Cox Regression. However, when relaxing the linear assumption with neural networks (e.g., Cox-MLP (Katzman et al., 2018; Kvamme et al., 2019)), we lose the guaranteed coverage. To recover the guaranteed coverage without linear assumption, we propose two algorithms based on conformal inference. In the first …
Investigating Data Usage for Inductive Conformal Predictors
针对归纳共形预测器在数据有限或成本高场景下的高效数据划分开放问题，通过多组实验探究划分方式及训练与校准集样本重叠的情况，为相关学术与实践应用提供参考。
Improving the statistical efficiency of cross-conformal prediction
本研究针对交叉共形预测及其简单变体，基于利用可交换性与随机化的高效p值组合结果提出新变体，在保留原有覆盖率理论保证的同时缩小预测集，模拟验证了结论并揭示相关权衡。
We propose a nonparametric quantile regression method using deep neural networks with a rectified linear unit penalty function to avoid quantile crossing. This penalty function is computationally feasible for enforcing non-crossing constraints in multi-dimensional nonparametric quantile regression. We establish non-asymptotic upper bounds for the excess risk of the proposed nonparametric quantile regression function estimators. Our error bounds achieve optimal minimax rate of convergence for the Holder class, and the prefactors of the error bounds depend polynomially on the dimension of the pr…
Forecasting Extreme High Summer Temperatures in Paris and Cairo Using Gradient Boosting and Conformal Prediction Regions
本文针对巴黎与开罗夏季极端高温，采用滞后14天的梯度提升模型提前两周预测气温和热指数的 Q(.95)，并以保形预测区域量化不确定性。
Exact and Approximate Conformal Inference for Multi-Output Regression
针对多输出回归中共形推断计算成本高的问题，推导了线性预测模型下共形推断p值的精确形式，并提出unionCP与多元rootCP两种高效近似方法，经理论和实验验证有效。
SpeedCP: Fast Kernel-based Conditional Conformal Prediction
提出一种基于路径追踪的高效算法，在保持RKHS条件共形预测框架理论优势的同时，将计算速度提升40倍，区间长度缩短30%。
Neural networks are among the most powerful nonlinear models used to address supervised learning problems. Similar to most machine learning algorithms, neural networks produce point predictions and do not provide any prediction interval which includes an unobserved response value with a specified probability. In this paper, we proposed the $k$-fold prediction interval method to construct prediction intervals for neural networks based on $k$-fold cross validation. Simulation studies and analysis of 10 real datasets are used to compare the finite-sample properties of the prediction intervals pro…
We develop a general framework for constructing distribution-free prediction intervals for time series. Theoretically, we establish explicit bounds on conditional and marginal coverage gaps of estimated prediction intervals, which asymptotically converge to zero under additional assumptions. We obtain similar bounds on the size of set differences between oracle and estimated prediction intervals. Methodologically, we introduce a computationally efficient algorithm called \texttt{EnbPI} that wraps around ensemble predictors, which is closely related to conformal prediction (CP) but does not req…
Conformal predictive systems are a recent modification of conformal predictors that output, in regression problems, probability distributions for labels of test observations rather than set predictions. The extra information provided by conformal predictive systems may be useful, e.g., in decision making problems. Conformal predictive systems inherit the relative computational inefficiency of conformal predictors. In this paper we discuss two computationally efficient versions of conformal predictive systems, which we call split conformal predictive systems and cross-conformal predictive syste…
We propose a robust method for constructing conditionally valid prediction intervals based on models for conditional distributions such as quantile and distribution regression. Our approach can be applied to important prediction problems including cross-sectional prediction, k-step-ahead forecasts, synthetic controls and counterfactual prediction, and individual treatment effects prediction. Our method exploits the probability integral transform and relies on permuting estimated ranks. Unlike regression residuals, ranks are independent of the predictors, allowing us to construct conditionally …
Selecting informative conformal prediction sets with false coverage rate control
本文提出 InfoSP 与 InfoSCOP，在分类和回归中经规则选择小而有信息量的保形预测集，并对所选样本实现有限样本 FCR 控制。
Multi-Scale Conformal Prediction: A Theoretical Framework with Coverage Guarantees
本文提出多尺度共形预测框架，通过在各尺度定义符合性函数并相交预测集，在保持有限样本覆盖保证的同时生成更小更精确的预测集。
Prediction, where observed data is used to quantify uncertainty about a future observation, is a fundamental problem in statistics. Prediction sets with coverage probability guarantees are a common solution, but these do not provide probabilistic uncertainty quantification in the sense of assigning beliefs to relevant assertions about the future observable. Alternatively, we recommend the use of a {\em probabilistic predictor}, a data-dependent (imprecise) probability distribution for the to-be-predicted observation given the observed data. It is essential that the probabilistic predictor be r…
This paper studies nonparametric regression with long memory (LRD) errors and predictors. First, we formulate general conditions which guarantee the standard rate of convergence for a nonparametric kernel estimator. Second, we calculate the Mean Integrated Squared Error (MISE). In particular, we show that LRD of errors may influence MISE. On the other hand, an estimator for a shape function is typically not influenced by LRD in errors. Finally, we investigate properties of a data-driven bandwidth choice. We show that Averaged Squared Error (ASE) is a good approximation of MISE, however, this i…
Reliable Statistical Guarantees for Conformal Predictors with Small Datasets
本文提出了一种新的统计保证方法，用于在小数据集上提高符合预测器的可靠性，通过提供覆盖概率信息并验证其有效性。
Conformal Robustness in Prediction-Driven Decision-Making
本文提出基于共形分数的鲁棒性框架，将黑箱预测器转化为决策相关的不确定性表示，统一可靠性与目标导向优化，实现可靠性保证和脆弱性分析。
In this article, we derive concentration inequalities for the cross-validation estimate of the generalization error for stable predictors in the context of risk assessment. The notion of stability has been first introduced by \cite{DEWA79} and extended by \cite{KEA95}, \cite{BE01} and \cite{KUNIY02} to characterize class of predictors with infinite VC dimension. In particular, this covers $k$-nearest neighbors rules, bayesian algorithm (\cite{KEA95}), boosting,... General loss functions and class of predictors are considered. We use the formalism introduced by \cite{DUD03} to cover a large var…
扫码添加微信好友，提出您的宝贵建议 👇
💡 备注请填写：网站反馈
发表机构由 AI 从论文文本抽取，如发现错误，欢迎反馈修正建议。