arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2206.07948cs.AIcs.HCcs.LG
收藏

形成有效的人工智能团队:构建机器学习模型,补充多个专家的能力

Forming Effective Human-AI Teams: Building Machine Learning Models that Complement the Capabilities of Multiple Experts

Karlsruhe Institute of Technology(卡尔斯鲁厄理工学院);GESIS - Leibniz Institute for the Social Sciences(GESIS——莱布尼茨社会科学研究所)

Patrick Hemmer, Sebastian Schellhammer, Michael Vössing, Johannes Jakubik, Gerhard Satzger

摘要:

机器学习(ML)模型越来越多地用于经常涉及与人类专家一起工作的应用领域。在这种情况下,当ML模型难以预测某些实例时,将其推迟到单个人类专家可能是有利的。虽然以前的工作集中在一个不同的人类专家的情况下,在许多现实世界的情况下,可能有几个人类专家具有不同的能力。在这项工作中,我们提出了一种训练分类模型的方法,以补充多个人类专家的能力。通过与分配系统一起联合训练分类器,分类器学会准确预测人类专家难以预测的实例,而分配系统学会将每个实例传递给最合适的团队成员-分类器或人类专家之一。我们评估我们提出的方法在多个实验中的公共数据集与“合成”专家和现实世界的医疗数据集注释多个放射科医生。我们的方法优于以前的工作,比最好的人类专家或分类器更准确。此外,它可以灵活地适应不同规模和不同专家多样性水平的团队。

英文摘要:

Machine learning (ML) models are increasingly being used in application domains that often involve working together with human experts. In this context, it can be advantageous to defer certain instances to a single human expert when they are difficult to predict for the ML model. While previous work has focused on scenarios with one distinct human expert, in many real-world situations several human experts with varying capabilities may be available. In this work, we propose an approach that trains a classification model to complement the capabilities of multiple human experts. By jointly training the classifier together with an allocation system, the classifier learns to accurately predict those instances that are difficult for the human experts, while the allocation system learns to pass each instance to the most suitable team member -- either the classifier or one of the human experts. We evaluate our proposed approach in multiple experiments on public datasets with "synthetic" experts and a real-world medical dataset annotated by multiple radiologists. Our approach outperforms prior work and is more accurate than the best human expert or a classifier. Furthermore, it is flexibly adaptable to teams of varying sizes and different levels of expert diversity.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑