arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2202.11812cs.HCcs.AI
收藏

招聘中人机合作的绩效和偏见调查

Investigations of Performance and Bias in Human-AI Teamwork in Hiring

Microsoft Research(微软研究院)

Andi Peng, Besmira Nushi, Emre Kiciman, Kori Inkpen, Ece Kamar

摘要:

在人工智能辅助决策中,有效的混合(人类-人工智能)团队合作不仅取决于人工智能的表现,还取决于其对人类决策的影响。虽然先前的工作研究了模型准确性对人类的影响,但我们在这里努力研究模型的预测性能和偏差如何在简化辅助决策任务中转移到人类的复杂动态。我们考虑了ML辅助招聘领域,在这种情况下,在受约束的选择环境中操作的人类可以选择他们是否希望利用经过训练的模型的推断来帮助从书面传记中选择候选人。我们进行了一项大规模的用户研究,利用了之前工作中重新创建的真实BIOS数据集,其中人类在有和没有三种不同的NLP分类器(随机,词袋和深度神经网络)的帮助下预测给定候选人的地面实况。我们的研究结果表明,虽然高性能模型显著提高了人类在混合环境中的表现,但有些模型减轻了混合偏见,而另一些模型则加剧了这种偏见。我们通过决策一致性的视角来研究这些发现,并观察到我们的模型架构选择对人类-AI一致性和偏见产生了影响,从而激发了在部署之前评估这些复杂动态的明确需求。

英文摘要:

In AI-assisted decision-making, effective hybrid (human-AI) teamwork is not solely dependent on AI performance alone, but also on its impact on human decision-making. While prior work studies the effects of model accuracy on humans, we endeavour here to investigate the complex dynamics of how both a model's predictive performance and bias may transfer to humans in a recommendation-aided decision task. We consider the domain of ML-assisted hiring, where humans -- operating in a constrained selection setting -- can choose whether they wish to utilize a trained model's inferences to help select candidates from written biographies. We conduct a large-scale user study leveraging a re-created dataset of real bios from prior work, where humans predict the ground truth occupation of given candidates with and without the help of three different NLP classifiers (random, bag-of-words, and deep neural network). Our results demonstrate that while high-performance models significantly improve human performance in a hybrid setting, some models mitigate hybrid bias while others accentuate it. We examine these findings through the lens of decision conformity and observe that our model architecture choices have an impact on human-AI conformity and bias, motivating the explicit need to assess these complex dynamics prior to deployment.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑