分类下的微分学习
Differentiable Learning Under Triage
摘要：
多种证据表明，预测模型可能受益于算法分类。在算法分类下，预测模型不会预测所有实例，而是将其中一些实例推迟给人类专家。然而，在算法分类下，模型的预测准确性和人类专家之间的相互作用还没有得到很好的理解。在这项工作中，我们首先正式描述了在何种情况下预测模型可以从算法分类中受益。在这样做的过程中，我们还证明了为全自动化训练的模型在分流下可能是次优的。然后，给定任何模型和所需的分流水平，我们表明，最佳的分流政策是一个确定性的阈值规则，其中分流决策是确定性的阈值模型和人为错误之间的差异，每个实例的水平。在这些结果的基础上，我们引入了一个实用的基于梯度的算法，保证找到一个序列的分流政策和预测模型的性能不断提高。使用来自两个重要应用-内容审核和科学发现-的合成和真实数据对各种监督学习任务进行的实验说明了我们的理论结果，并表明我们基于梯度的算法提供的模型和分类策略优于几个竞争基线提供的模型和分类策略。
英文摘要：
Multiple lines of evidence suggest that predictive models may benefit from algorithmic triage. Under algorithmic triage, a predictive model does not predict all instances but instead defers some of them to human experts. However, the interplay between the prediction accuracy of the model and the human experts under algorithmic triage is not well understood. In this work, we start by formally characterizing under which circumstances a predictive model may benefit from algorithmic triage. In doing so, we also demonstrate that models trained for full automation may be suboptimal under triage. Then, given any model and desired level of triage, we show that the optimal triage policy is a deterministic threshold rule in which triage decisions are derived deterministically by thresholding the difference between the model and human errors on a per-instance level. Building upon these results, we introduce a practical gradient-based algorithm that is guaranteed to find a sequence of triage policies and predictive models of increasing performance. Experiments on a wide variety of supervised learning tasks using synthetic and real data from two important applications -- content moderation and scientific discovery -- illustrate our theoretical results and show that the models and triage policies provided by our gradient-based algorithm outperform those provided by several competitive baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
