Paris-Saclay University(巴黎萨克雷大学);CNRS(法国国家科学研究中心);University of California, Berkeley(加州大学伯克利分校);Laboratoire Interdisciplinaire des Sciences du Numerique(数字科学跨学科实验室);Capgemini Invent France(凯捷创新法国公司);ENSIIE(法国国立统计与信息工程学院);INRIA Paris(法国国家信息与自动化研究所巴黎分部)
Vincent Blot, Anastasios N Angelopoulos, Michael I Jordan, Nicolas J-B Brunel
Science and technology have a growing need for effective mechanisms that ensure reliable, controlled performance from black-box machine learning algorithms. These performance guarantees should ideally hold conditionally on the input-that is the performance guarantees should hold, at least approximately, no matter what the input. However, beyond stylized discrete groupings such as ethnicity and gender, the right notion of conditioning can be difficult to define. For example, in problems such as image segmentation, we want the uncertainty to reflect the intrinsic difficulty of the test sample, but this may be difficult to capture via a conditioning event. Building on the recent work of Gibbs et al. [2023], we propose a methodology for achieving approximate conditional control of statistical risks-the expected value of loss functions-by adapting to the difficulty of test samples. Our framework goes beyond traditional conditional risk control based on user-provided conditioning events to the algorithmic, data-driven determination of appropriate function classes for conditioning. We apply this framework to various regression and segmentation tasks, enabling finer-grained control over model performance and demonstrating that by continuously monitoring and adjusting these parameters, we can achieve superior precision compared to conventional risk-control methods.
Department of Computer Science, New Jersey Institute of Technology(计算机科学系,新泽西理工学院);Department of Mathematical Sciences, New Jersey Institute of Technology(数学科学系,新泽西理工学院)
Confident Object Detection via Conformal Prediction and Conformal Risk Control: an Application to Railway Signaling
Institut de Mathématiques de Toulouse(图卢兹数学研究所);SNCF(法国国家铁路公司);Brown University(布朗大学);Thales AVS France SAS(泰雷兹航空航天系统法国股份有限公司);IRT Saint Exupéry(圣埃克苏佩里技术研究所)
Reliable Real-Time Value at Risk Estimation via Quantile Regression Forest with Conformal Calibration
Institute of Statistics and Big Data, Renmin University of China(统计与大数据研究院,中国人民大学);Department of Decision Analytics and Operations, City University of Hong Kong(决策分析与运营部门,香港城市大学);School of Information, Renmin University of China(信息学院,中国人民大学)
Bayesian Conformal Prediction as a Decision Risk Problem
Department of Computer Science, University of Manchester(曼彻斯特大学计算机科学系);UKRI AI Centre for Doctoral Training in Decision Making for Complex Systems(UKRI人工智能博士培训中心(复杂系统决策));Department of Computer Science, Aalto University(艾洛大学计算机科学系);ELLIS Institute, Finland(芬兰ELLIS研究所)
Division of Systems and Control, Department of Information Technology, Uppsala University, Sweden(信息科技系系统与控制 division,乌普萨拉大学,瑞典);Science for Life Laboratory, Uppsala, Sweden(生命科学实验室,乌普萨拉,瑞典)
Instituto de Telecomunicações(电信研究所);Instituto Superior Técnico, Universidade de Lisboa(里斯本大学高等技术学院);IT University of Copenhagen(哥本哈根信息技术大学);Pioneer Centre for Artificial Intelligence(先锋人工智能中心);Unbabel(Unbabel公司)
Decoupled Conformal Optimisation: Efficient Prediction Sets via Independent Tuning and Calibration
Department of Computer Science, University of Manchester(曼彻斯特大学计算机科学系);UKRI AI Centre for Doctoral Training in Decision Making for Complex Systems(UKRI人工智能博士培训中心(复杂系统决策));Department of Computer Science, Aalto University(艾尔沃斯大学计算机科学系);ELLIS Institute, Finland(芬兰ELLIS研究所)
A Deployment Audit of Release-Side Risk in Conformal Triage under Prevalence Shift
University of Illinois Chicago(伊利诺伊大学芝加哥分校);Manteia Technologies Co., Ltd(Manteia技术有限公司);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of California Los Angeles(加州大学洛杉矶分校)
Aligning Model Properties via Conformal Risk Control
Stanford University(斯坦福大学);Graduate School of Business, Stanford University(斯坦福大学商学院);Management Science and Engineering, Stanford University(斯坦福大学管理科学与工程学院)