arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.19494cs.LGstat.ML
收藏

多专家延迟的回归

Regression with Multi-Expert Deferral

Courant Institute of Mathematical Sciences(柯朗数学科学研究所);Google Research(谷歌研究院)

Anqi Mao, Mehryar Mohri, Yutao Zhong

AI总结:

本文提出多专家延迟回归框架,设计单阶段和两阶段方法及具有$H$-一致性界保证的替代损失函数,并通过实验验证其有效性。

摘要:

多专家学习延迟是一个框架,其中学习者可以选择将预测延迟给多个专家。虽然该问题在分类场景中已受到大量关注,但由于标签空间的无限和连续性质,它在回归中呈现出独特的挑战。在这项工作中,我们引入了一个新颖的带延迟的回归框架,该框架涉及将预测延迟给多个专家。我们对单阶段场景和两阶段场景都进行了全面分析,其中单阶段场景同时学习预测器和延迟函数,两阶段场景则涉及一个预训练的预测器和一个学习得到的延迟函数。我们为这两种场景引入了新的替代损失函数,并证明它们由$H$-一致性界所支持。这些界提供了一致性保证,比贝叶斯一致性更强,因为它们是渐近无关的且针对特定假设集。我们的框架具有通用性,适用于多个专家,可容纳任何有界回归损失,同时处理实例依赖和标签依赖的成本,并支持单阶段和两阶段方法。一个副产品是,我们的单阶段公式包含了最近的带弃权(不执行)的回归框架(Cheng et al., 2023)作为特例,其中仅考虑单个专家、平方损失和标签无关成本。直接最小化我们提出的损失函数可得到用于带延迟回归的新算法。我们报告了大量实验的结果,表明了我们提出的算法的有效性。

英文摘要:

Learning to defer with multiple experts is a framework where the learner can choose to defer the prediction to several experts. While this problem has received significant attention in classification contexts, it presents unique challenges in regression due to the infinite and continuous nature of the label space. In this work, we introduce a novel framework of regression with deferral, which involves deferring the prediction to multiple experts. We present a comprehensive analysis for both the single-stage scenario, where there is simultaneous learning of predictor and deferral functions, and the two-stage scenario, which involves a pre-trained predictor with a learned deferral function. We introduce new surrogate loss functions for both scenarios and prove that they are supported by $H$-consistency bounds. These bounds provide consistency guarantees that are stronger than Bayes consistency, as they are non-asymptotic and hypothesis set-specific. Our framework is versatile, applying to multiple experts, accommodating any bounded regression losses, addressing both instance-dependent and label-dependent costs, and supporting both single-stage and two-stage methods. A by-product is that our single-stage formulation includes the recent regression with abstention framework (Cheng et al., 2023) as a special case, where only a single expert, the squared loss and a label-independent cost are considered. Minimizing our proposed loss functions directly leads to novel algorithms for regression with deferral. We report the results of extensive experiments showing the effectiveness of our proposed algorithms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑