arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.10909cs.AIcs.HC
收藏

细粒度适当依赖:面向复杂任务分解的多步透明决策流程中的人机协作

Fine-Grained Appropriate Reliance: Human-AI Collaboration with a Multi-Step Transparent Decision Workflow for Complex Task Decomposition

Delft University of Technology(代尔夫特理工大学);Karlsruhe Institute of Technology(卡尔斯鲁厄理工学院)

Gaole He, Patrick Hemmer, Michael Vössing, Max Schemmer, Ujwal Gadiraju

AI总结:

本研究提出多步透明决策工作流,通过实证(N=233)表明其在复合事实核查任务中可提升人机协作适当依赖,尤其在AI建议误导时优于单步协作,强调无通用工作流。

摘要:

近年来,人工智能系统的快速发展带来了智能服务的益处,但也引发了关于安全性和可靠性的担忧。通过培养用户对人工智能系统的适当依赖,可以实现互补的团队绩效和减少人类工作量。以往的实证研究广泛分析了从任务、系统到人类行为等因素对用户信任和适当依赖的影响,但研究背景多局限于单步决策情境。然而,在需要多步工作流的复杂语义任务中,用户对人工智能系统的依赖仍未得到充分探索。受近期基于大语言模型的任务分解研究的启发,我们提出研究一种新颖的多步透明(MST)决策工作流对用户依赖行为的影响。我们开展了一项实证研究(N = 233),涉及复合事实核查任务(即包含多个子事实验证步骤的事实核查任务)中的人工智能辅助决策。我们的研究结果表明,在特定情境下(例如,当人工智能系统的建议具有误导性时),采用MST决策工作流的人机协作可以优于单步协作。对细粒度层面适当依赖的进一步分析表明,当用户对中间步骤表现出相对较高的考量时,MST决策工作流可能有效。我们的工作强调,不存在一种放之四海而皆准的决策工作流能够帮助实现最优的人机协作。我们的见解有助于加深对决策工作流在促进适当依赖方面作用的理解。我们综合了在设计有效手段以促进复合任务中对人工智能系统适当依赖方面的重要启示,为人本人工智能和更广泛的人机交互社区提供了机遇。

英文摘要:

In recent years, the rapid development of AI systems has brought about the benefits of intelligent services but also concerns about security and reliability. By fostering appropriate user reliance on an AI system, both complementary team performance and reduced human workload can be achieved. Previous empirical studies have extensively analyzed the impact of factors ranging from task, system, and human behavior on user trust and appropriate reliance in the context of one-step decision making. However, user reliance on AI systems in tasks with complex semantics that require multi-step workflows remains under-explored. Inspired by recent work on task decomposition with large language models, we propose to investigate the impact of a novel Multi-Step Transparent (MST) decision workflow on user reliance behaviors. We conducted an empirical study (N = 233) of AI-assisted decision making in composite fact-checking tasks (i.e., fact-checking tasks that entail multiple sub-fact verification steps). Our findings demonstrate that human-AI collaboration with an MST decision workflow can outperform one-step collaboration in specific contexts (e.g., when advice from an AI system is misleading). Further analysis of the appropriate reliance at fine-grained levels indicates that an MST decision workflow can be effective when users demonstrate a relatively high consideration of the intermediate steps. Our work highlights that there is no one-size-fits-all decision workflow that can help obtain optimal human-AI collaboration. Our insights help deepen the understanding of the role of decision workflows in facilitating appropriate reliance. We synthesize important implications for designing effective means to facilitate appropriate reliance on AI systems in composite tasks, positioning opportunities for the human-centered AI and broader HCI communities.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑