arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2301.06197cs.LGcs.HC
收藏

谁应进行预测?学习向人类推迟决策的精确算法

Who Should Predict? Exact Algorithms For Learning to Defer to Humans

MIT-IBM Watson AI Lab(麻省理工学院-IBM沃森人工智能实验室);Massachusetts Institute of Technology(麻省理工学院);IBM Research(IBM研究院)

Hussein Mozannar, Hunter Lang, Dennis Wei, Prasanna Sattigeri, Subhro Das, David Sontag

AI总结:

本文研究学习向人类推迟决策的问题,证明现有方法在可实现设定下仍可能失败且该问题为NP难,并提出MILP精确求解公式及一种可实现一致的替代损失函数,在多个数据集上验证了有效性。

摘要:

自动化AI分类器应能够将预测推迟给人类决策者,以确保更准确的预测。在这项工作中,我们联合训练一个分类器和一个拒绝器(rejector),由拒绝器在每个数据点上决定是由分类器还是由人类进行预测。我们证明,即使存在具有零误差的线性分类器和拒绝器(可实现设定),先前的方法也可能无法找到具有低误分类误差的人机AI系统。我们证明,即使在问题可实现的情况下,获得低误差的线性组合也是NP难的。为了补充这一负面结果,我们给出了一种混合整数线性规划(MILP)公式,可以在线性设定下最优地解决该问题。然而,MILP仅能扩展到中等规模的问题。因此,我们提供了一种新的替代损失函数,该函数具有可实现一致性,并在经验上表现良好。我们在一个全面的数据集集合上测试了我们的方法,并与广泛的基线方法进行了比较。

英文摘要:

Automated AI classifiers should be able to defer the prediction to a human decision maker to ensure more accurate predictions. In this work, we jointly train a classifier with a rejector, which decides on each data point whether the classifier or the human should predict. We show that prior approaches can fail to find a human-AI system with low misclassification error even when there exists a linear classifier and rejector that have zero error (the realizable setting). We prove that obtaining a linear pair with low error is NP-hard even when the problem is realizable. To complement this negative result, we give a mixed-integer-linear-programming (MILP) formulation that can optimally solve the problem in the linear setting. However, the MILP only scales to moderately-sized problems. Therefore, we provide a novel surrogate loss function that is realizable-consistent and performs well empirically. We test our approaches on a comprehensive set of datasets and compare to a wide range of baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑