Learning algorithms are often used in conjunction with expert decision makers in practical scenarios, however this fact is largely ignored when designing these algorithms. In this paper we explore how to learn predictors that can either predict or choose to defer the decision to a downstream expert. Given only samples of the expert's decisions, we give a procedure based on learning a classifier and a rejector and analyze it theoretically. Our approach is based on a novel reduction to cost sensitive learning where we give a consistent surrogate loss for cost sensitive learning that generalizes the cross entropy loss. We show the effectiveness of our approach on a variety of experimental tasks.
Why Ask One When You Can Ask $k$? Learning-to-Defer to the Top-$k$ Experts
School of Computing, National University of Singapore(新加坡国立大学计算机学院);Fédération ENAC, ISAE-SUPAERO, ONERA, Université de Toulouse(ENAC联合会、ISAE-SUPAERO、ONERA、图卢兹大学);Agency for Science, Technology and Research, Institute for Infocomm Research(科技研究局、信息通信研究所)
School of Computing, National University of Singapore(新加坡国立大学计算机学院);Fédération ENAC ISAE-SUPAERO ONERA, Université de Toulouse, France(法国图卢兹大学ENAC ISAE-SUPAERO ONERA联合体);Institute for Infocomm Research, A*STAR , Singapore(新加坡A*STAR信息通信研究所);IPAL, IRL 2955, Singapore(新加坡IPAL实验室);Department of Mathematics, National University of Singapore(新加坡国立大学数学系)
Learning to Defer for Causal Discovery with Imperfect Experts
University of Oxford(牛津大学);Mila – Quebec Artificial Intelligence Institute(米拉-魁北克人工智能研究所);Université de Montréal(蒙特利尔大学);ServiceNow Research(ServiceNow研究院);University of Amsterdam(阿姆斯特丹大学);Université Laval(拉瓦尔大学)
Consistent Learning-to-Defer with Expert-Conditional Advice
School of Computing(计算学院);Département de Mathématiques(数学系);National University of Singapore(新加坡国立大学);Sorbonne University(索邦大学);Fédération ENAC(ENAC联合会);ISAE-SUPAERO;ONERA;Agency for Science, Technology and Research(科技研究局);Université de Toulouse(图卢兹大学);Institute for Infocomm Research(信息与通信研究所)
Beyond Augmented-Action Surrogates for Multi-Expert Learning-to-Defer
School of Computing(计算学院);Fédération ENAC(ENAC联合会);ISAE-SUPAERO;ONERA;National University of Singapore(新加坡国立大学);Université de Toulouse(图卢兹大学);Agency for Science, Technology and Research(科技研究局);Institute for Infocomm Research(信息通信研究所)
Learning-to-Defer in Non-Stationary Time Series via Switching State-Space Models
School of Computing(计算机科学学院);National University of Singapore(新加坡国立大学);Institute for Infocomm Research(信息通信研究所);ISAE-SUPAERO;ONERA;A*STAR, Singapore(新加坡A*STAR)
Adversarial Robustness in Two-Stage Learning-to-Defer: Algorithms and Guarantees
School of Computing, National University of Singapore, Singapore(新加坡国立大学计算机学院);IRIT, Université de Toulouse, CNRS, Toulouse INP, Toulouse, France(图卢兹大学IRIT研究所、法国国家科学研究中心、图卢兹INP);Institute for Infocomm Research, Agency for Science, Technology and Research, Singapore(新加坡科技研究局信息与通信研究所)
Adversarial Robustness in One-Stage Learning-to-Defer
School of Computing, National University of Singapore(新加坡国立大学计算机学院);Fédération ENAC ISAE-SUPAERO ONERA, Université de Toulouse, France(法国图卢兹大学ENAC ISAE-SUPAERO ONERA联合体);Institute for Infocomm Research, A*STAR , Singapore(新加坡星展研究所);IPAL, IRL 2955, Singapore(新加坡IPAL实验室)
Incorporating Uncertainty in Learning to Defer Algorithms for Safe Computer-Aided Diagnosis
Centre for Big Data Research in Health, University of New South Wales(新南威尔士大学健康大数据研究中心)
深度神经网络越来越多地用于计算机辅助诊断,但错误的诊断可能会给患者带来极其高昂的代价。我们提出了一个学习推迟与不确定性(LDU)算法,识别患者的诊断不确定性高,并推迟他们的评估由人类专家。评价LDU对心肌梗死的诊断(使用出院总结)、任何合并症的诊断(使用结构化数据)以及胸腔积液和气胸的诊断(使用胸部X线片),并与“学习在没有不确定信息的情况下推迟”(LD)和“不确定性直接分诊”(DT)方法进行比较。LDU达到了与LD相同的F1评分,但延迟的患者数量显著减少(例如,对于诊断胸腔积液,F1评分为0.96,延迟率为36% vs. 69%)。此外,即使许多患者被分配了具有高置信度的错误诊断(例如,对于任何合并症的诊断),LDU的F1评分也增加了17%,而DT不适用。重要的是,LDU中延迟损失的权重可以很容易地调整,以获得诊断准确性和延迟率之间的理想权衡。总之,LDU可以很容易地增强任何现有的诊断网络,以降低临床实践中误诊的风险。
A2C: A Modular Multi-stage Collaborative Decision Framework for Human-AI Teams
Institute for Clarity in Documentation(文献清晰度研究所);Inria Paris-Rocquencourt(法国国家信息与自动化研究所巴黎-罗康库尔中心);Rajiv Gandhi University(拉吉夫·甘地大学);Tsinghua University(清华大学);Palmer Research Laboratories(帕尔默研究实验室)
Online Irregular Multivariate Time Series Forecasting via Uncertainty-Driven Dual-Expert Calibration
Key Laboratory of Big Data \& Artificial Intelligence in Transportation (Beijing Jiaotong University), Ministry of Education School of Computer Science;Technology, Beijing Jiaotong University Beijing China;School of Computer Science;Tangshan Research Institute, Beijing Jiaotong University Tangshan China;Key Laboratory of Big Data \& Artificial Intelligence in Transportation (Beijing Jiaotong University), Ministry of Education;Technology, Beijing Jiaotong University;Tangshan Research Institute, Beijing Jiaotong University
PrismFlow: Residual Dynamics for Flow Matching in Time-Series Generation
Zhejiang University, China(浙江大学,中国);Nanyang Technological University, Singapore(南洋理工大学,新加坡);I2R, Agency for Science, Technology and Research (A*STAR), Singapore(科技研究局(A*STAR)新加坡研究所,新加坡)
A Stochastic Path-Integrated Differential EstimatoR Expectation Maximization Algorithm
Institut de Mathématiques de Toulouse(图卢兹数学研究所);Université de Toulouse(图卢兹大学);CNRS(法国国家科学研究中心);Centre de Mathématiques Appliquées(应用数学中心);Ecole Polytechnique(巴黎综合理工学院);HSE University(高等经济大学);The Chinese University of Hong Kong(香港中文大学)