arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2208.07960cs.HCcs.AI
收藏

推进人类-人工智能互补性:用户专业知识和数学调整对联合决策的影响

Advancing Human-AI Complementarity: The Impact of User Expertise and Algorithmic Tuning on Joint Decision Making

Microsoft Research(微软研究院);University of Virginia(弗吉尼亚大学);University of Washington(华盛顿大学);University of Michigan(密歇根大学);Human Computation Institute(人类计算研究所);LMU Munich(慕尼黑大学);Western Washington University(西华盛顿大学)

Kori Inkpen, Shreya Chappidi, Keri Mallari, Besmira Nushi, Divya Ramesh, Pietro Michelucci, Vani Mandava, Libuše Hannah Vepřek, Gabrielle Quinn

摘要:

人类-人工智能协作决策努力实现团队绩效,超过人类或人工智能单独的绩效。然而,许多因素可能会影响人类-AI团队的成功,包括用户的领域专业知识,AI系统的心理模型,对建议的信任等等。这项工作研究了用户与三个模拟算法模型的交互,所有这些模型都具有相似的准确性,但对真阳性率和真阴性率的调整不同。我们的研究检查了用户在一个重要的血管标记任务中的表现,参与者指出了给定的血管是流动还是停滞。我们的研究结果表明,虽然来自AI助手的建议可以帮助用户决策,但用户相对于AI的基线性能和AI错误类型的补充调整等因素会显著影响整体团队绩效。新手用户有所改善,但没有达到AI的准确性水平。高度熟练的用户通常能够辨别何时应该遵循人工智能的建议,并通常保持或提高他们的性能。与人工智能具有相似准确性水平的中等表现者,在人工智能建议是否有助于或损害他们的表现方面变化最大。此外,我们发现,用户对人工智能性能的感知相对于他们自己也对他们在获得人工智能建议时的准确性是否提高产生了重大影响。这项工作提供了有关人机协作因素复杂性的见解,并就如何开发以人为本的人工智能算法以补充决策任务中的用户提供了建议。

英文摘要:

Human-AI collaboration for decision-making strives to achieve team performance that exceeds the performance of humans or AI alone. However, many factors can impact success of Human-AI teams, including a user's domain expertise, mental models of an AI system, trust in recommendations, and more. This work examines users' interaction with three simulated algorithmic models, all with similar accuracy but different tuning on their true positive and true negative rates. Our study examined user performance in a non-trivial blood vessel labeling task where participants indicated whether a given blood vessel was flowing or stalled. Our results show that while recommendations from an AI-Assistant can aid user decision making, factors such as users' baseline performance relative to the AI and complementary tuning of AI error types significantly impact overall team performance. Novice users improved, but not to the accuracy level of the AI. Highly proficient users were generally able to discern when they should follow the AI recommendation and typically maintained or improved their performance. Mid-performers, who had a similar level of accuracy to the AI, were most variable in terms of whether the AI recommendations helped or hurt their performance. In addition, we found that users' perception of the AI's performance relative on their own also had a significant impact on whether their accuracy improved when given AI recommendations. This work provides insights on the complexity of factors related to Human-AI collaboration and provides recommendations on how to develop human-centered AI algorithms to complement users in decision-making tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑