Many real-world multi-label prediction problems involve set-valued predictions that must satisfy specific requirements dictated by downstream usage. We focus on a typical scenario where such requirements, separately encoding $\textit{value}$ and $\textit{cost}$, compete with each other. For instance, a hospital might expect a smart diagnosis system to capture as many severe, often co-morbid, diseases as possible (the value), while maintaining strict control over incorrect predictions (the cost). We present a general pipeline, dubbed as FavMac, to maximize the value while controlling the cost in such scenarios. FavMac can be combined with almost any multi-label classifier, affording distribution-free theoretical guarantees on cost control. Moreover, unlike prior works, it can handle real-world large-scale applications via a carefully designed online update mechanism, which is of independent interest. Our methodological and theoretical contributions are supported by experiments on several healthcare tasks and synthetic datasets - FavMac furnishes higher value compared with several variants and baselines while maintaining strict cost control. Our code is available at https://github.com/zlin7/FavMac
School of Statistics and Data Science, LPMC, KLMDASR and LEBPS, Nankai University(南开大学统计与数据科学学院);Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院)
Mirror Online Conformal Prediction with Intermittent Feedback
King’s Communications, Learning and Information Processing (KCLIP) Lab, Department of Engineering, King’s College London(国王学院通信、学习和信息处理实验室(KCLIP)实验室,工程系,伦敦国王学院)
Parameter-Free and Group Conditional Online Conformal Prediction
Data Science and AI Institute (DSAI), Johns Hopkins University(数据科学与人工智能研究院(DSAI),约翰霍普金斯大学);Mathematical Institute for Data Science (MINDS), Johns Hopkins University(数据科学数学研究院(MINDS),约翰霍普金斯大学);Department of Biomedical Engineering, Johns Hopkins University(生物医学工程系,约翰霍普金斯大学);Department of Computer Science, Johns Hopkins University(计算机科学系,约翰霍普金斯大学);Amazon Responsible AI(亚马逊负责任人工智能)
Exploring the Noise Robustness of Online Conformal Prediction
Department of Statistics and Data Science, Southern University of Science and Technology(统计与数据科学系,南方科技大学);Department of Computer and Information Science, University of Macau(计算机与信息科学系,澳门大学);Center for Data Science, Zhejiang University(数据科学中心,浙江大学)
Reliable Real-Time Value at Risk Estimation via Quantile Regression Forest with Conformal Calibration
Institute of Statistics and Big Data, Renmin University of China(统计与大数据研究院,中国人民大学);Department of Decision Analytics and Operations, City University of Hong Kong(决策分析与运营部门,香港城市大学);School of Information, Renmin University of China(信息学院,中国人民大学)
A Profit-Maximizing Strategy for Advertising on the e-Commerce Platforms
South China University of Technology(华南理工大学);Guangdong University of Foreign Studies(广东外语外贸大学);ShenZhen TingZhu Health Management Co.,Ltd.(深圳市汀竹健康管理有限公司)
Online Conformal Anomaly Detection with Prediction-Powered Data Acquisition
King’s Communications, Learning & Information Processing (KCLIP) lab within the Centre for Intelligent Information Processing Systems (CIIPS), Department of Engineering, King’s College London(伦敦国王学院工程系智能信息处理系统中心(CIIPS)通信、学习与信息处理实验室(KCLIP))
Dynamic Estimation Loss Control in Variational Quantum Sensing via Online Conformal Inference
Signal Processing Systems Group, Department of Electrical Engineering, Eindhoven University of Technology(埃因霍温理工大学电子工程系信号处理系统组);KCLIP lab, Center for Intelligent Information Processing Systems, Department of Engineering, King’s College London(伦敦国王学院工程系智能信息处理系统中心)
Adaptive Subgradient Methods for Online AUC Maximization
The University of Chicago(芝加哥大学);Singapore Management University(新加坡管理大学);Institute for Infocomm Research, A*STAR(新加坡科技研究局资讯通信研究院);Nanyang Technological University(南洋理工大学)
Online Conformal Prediction with Corrupted Feedback
Department of Engineering, King’s College London(伦敦国王学院工程系);Communication Systems Department, EURECOM(EURECOM通信系统部);Institute for Intelligent Networked Systems, Northeastern University London(伦敦东北大学智能网络系统研究所)