We study the problem of conformal prediction in a novel online framework that directly optimizes efficiency. In our problem, we are given a target miscoverage rate $α> 0$, and a time horizon $T$. On each day $t \le T$ an algorithm must output an interval $I_t \subseteq [0, 1]$, then a point $y_t \in [0, 1]$ is revealed. The goal of the algorithm is to achieve coverage, that is, $y_t \in I_t$ on (close to) a $(1 - α)$-fraction of days, while maintaining efficiency, that is, minimizing the average volume (length) of the intervals played. This problem is an online analogue to the problem of constructing efficient confidence intervals.
We study this problem over arbitrary and exchangeable (random order) input sequences. For exchangeable sequences, we show that it is possible to construct intervals that achieve coverage $(1 - α) - o(1)$, while having length upper bounded by the best fixed interval that achieves coverage in hindsight. For arbitrary sequences however, we show that any algorithm that achieves a $μ$-approximation in average length compared to the best fixed interval achieving coverage in hindsight, must make a multiplicative factor more mistakes than $αT$, where the multiplicative factor depends on $μ$ and the aspect ratio of the problem. Our main algorithmic result is a matching algorithm that can recover all Pareto-optimal settings of $μ$ and number of mistakes. Furthermore, our algorithm is deterministic and therefore robust to an adaptive adversary.
This gap between the exchangeable and arbitrary settings is in contrast to the classical online learning problem. In fact, we show that no single algorithm can simultaneously be Pareto-optimal for arbitrary sequences and optimal for exchangeable sequences. On the algorithmic side, we give an algorithm that achieves the near-optimal tradeoff between the two cases.
School of Statistics and Data Science, LPMC, KLMDASR and LEBPS, Nankai University（南开大学统计与数据科学学院）；Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）
Conformal Prediction Intervals with Tail-Specific Guarantees
Sapienza University of Rome（罗马第一大学）；MEMOTEF Department, Sapienza University of Rome（罗马第一大学MEMOTEF系）；MRC–Biostatistics Unit, University of Cambridge（剑桥大学医学研究委员会生物统计单元）
Reliable Real-Time Value at Risk Estimation via Quantile Regression Forest with Conformal Calibration
Institute of Statistics and Big Data, Renmin University of China（统计与大数据研究院，中国人民大学）；Department of Decision Analytics and Operations, City University of Hong Kong（决策分析与运营部门，香港城市大学）；School of Information, Renmin University of China（信息学院，中国人民大学）
Parameter-Free and Group Conditional Online Conformal Prediction
Data Science and AI Institute (DSAI), Johns Hopkins University（数据科学与人工智能研究院（DSAI），约翰霍普金斯大学）；Mathematical Institute for Data Science (MINDS), Johns Hopkins University（数据科学数学研究院（MINDS），约翰霍普金斯大学）；Department of Biomedical Engineering, Johns Hopkins University（生物医学工程系，约翰霍普金斯大学）；Department of Computer Science, Johns Hopkins University（计算机科学系，约翰霍普金斯大学）；Amazon Responsible AI（亚马逊负责任人工智能）
ConfHit: Conformal Generative Design with Oracle Free Guarantees
Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校Siebel计算与数据科学学院）；Department of Statistics and Data Science, Wharton School, University of Pennysylvania（宾夕法尼亚大学沃顿商学院统计与数据科学系）
Efficient Federated Conformal Prediction with Group-Conditional Guarantee
IoT Thrust（物联网 thrust）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Institute for Intelligent Networked Systems (INSI)（智能网络化系统研究所）；Northeastern University London（伦敦东北大学）；Department of ECE The Hong Kong University of Science and Technology HK SAR（电子工程系 香港科技大学香港特别行政区）
Enhancing Adversarial Robustness with Conformal Prediction: A Framework for Guaranteed Model Reliability
Huaiyin Institute of Technology, Huai'an, China（淮阴职业技术学院）；Chengdu Research Institute, City University of Hong Kong, Chengdu, China（香港城市大学成都研究院）；Saarland University, Saarbrücken, Germany（萨尔兰大学）；Alpha Benito Research, Los Angeles, USA（Alpha Benito研究公司）；City University of Hong Kong, Hong Kong, China（香港城市大学）
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；ETH Zurich（苏黎世联邦理工学院）；Tsinghua University（清华大学）；HOCH Health Ostschweiz, Kantonsspital St. Gallen（瑞士东部健康组织，圣加仑州立医院）
