Conformal prediction is a learning framework controlling prediction coverage of prediction sets, which can be built on any learning algorithm for point prediction. This work proposes a learning framework named conformal loss-controlling prediction, which extends conformal prediction to the situation where the value of a loss function needs to be controlled. Different from existing works about risk-controlling prediction sets and conformal risk control with the purpose of controlling the expected values of loss functions, the proposed approach in this paper focuses on the loss for any test object, which is an extension of conformal prediction from miscoverage loss to some general loss. The controlling guarantee is proved under the assumption of exchangeability of data in finite-sample cases and the framework is tested empirically for classification with a class-varying loss and statistical postprocessing of numerical weather forecasting applications, which are introduced as point-wise classification and point-wise regression problems. All theoretical analysis and experimental results confirm the effectiveness of our loss-controlling approach.
Loss-Controlling Calibration with Asymptotic Guarantees
Tianjin University(天津大学);Shandong University of Technology(山东理工大学);Shandong University of Traditional Chinese Medicine(山东中医药大学);Hefei University of Technology(合肥工业大学)
Does confidence calibration improve conformal prediction?
Southern University of Science and Technology(南方科技大学);Nanyang Technological University(南洋理工大学);University of Macau(澳门大学);Singapore University of Technology and Design(新加坡科技设计大学)
Controlling Privacy Loss in Sampling Schemes: an Analysis of Stratified and Cluster Sampling
Boston University(波士顿大学);Institute for Employment Research(就业研究所);University of Maryland(马里兰大学);Apple(苹果公司);Harvard John A. Paulson School of Engineering and Applied Sciences(哈佛大学约翰·A·保尔森工程与应用科学学院)
Exploring the Noise Robustness of Online Conformal Prediction
Department of Statistics and Data Science, Southern University of Science and Technology(统计与数据科学系,南方科技大学);Department of Computer and Information Science, University of Macau(计算机与信息科学系,澳门大学);Center for Data Science, Zhejiang University(数据科学中心,浙江大学)
Denoised Conformal Alignment for Reliable Selection of Conditional Average Treatment Effect Predictions
Shanghai Jiao Tong University(上海交通大学);Shanghai University of Finance and Economics(上海财经大学);National University of Defense Technology(国防科学技术大学);School of Statistics and Data Science(统计与数据科学学院);Institute of Big Data Research(大数据研究院)
Dynamic Estimation Loss Control in Variational Quantum Sensing via Online Conformal Inference
Signal Processing Systems Group, Department of Electrical Engineering, Eindhoven University of Technology(埃因霍温理工大学电子工程系信号处理系统组);KCLIP lab, Center for Intelligent Information Processing Systems, Department of Engineering, King’s College London(伦敦国王学院工程系智能信息处理系统中心)
Instituto de Telecomunicações(电信研究所);Instituto Superior Técnico, Universidade de Lisboa(里斯本大学高等技术学院);IT University of Copenhagen(哥本哈根信息技术大学);Pioneer Centre for Artificial Intelligence(先锋人工智能中心);Unbabel(Unbabel公司)
Error-quantified Conformal Inference for Time Series
School of Statistics and Data Science, LPMC, KLMDASR and LEBPS, Nankai University(统计与数据科学学院,LPMC,KLMDASR和LEBPS,南开大学);Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院,清华大学)