谁先走?人工智能工作流程对临床成像决策的影响
Who Goes First? Influences of Human-AI Workflow on Decision Making in Clinical Imaging
摘要:
在人工智能技术的实际应用中,必须考虑支持人类与人工智能协作的设计和机制的细节。人工智能辅助人类决策的交互设计的一个关键方面是在更大的决策工作流中显示和排序人工智能推理的策略。我们对人工智能推理在人类审查手头的诊断任务之前和之后的影响了解甚少。我们探讨了在放射科诊断会议开始时提供AI帮助与放射科医生做出临时决定后的效果。我们进行了一项用户研究,其中19名兽医放射科医生在AI工具的帮助下识别了患者X射线图像中存在的放射学发现。我们采用了两种工作流程配置来分析(i)锚定效应,(ii)人类-AI团队诊断性能和一致性,(iii)决策所花费的时间和信心,以及(iv)AI的感知有用性。我们发现,被要求在审查AI推理之前注册临时响应的参与者不太可能同意AI,无论建议是否准确,并且在与AI不一致的情况下,不太可能寻求同事的第二意见。这些参与者还报告说,人工智能的建议不太有用。令人惊讶的是,要求在显示AI推理之前对案件做出临时决定并没有延长参与者在任务上花费的时间。该研究为临床AI工具在人在环系统中的部署提供了可推广和可操作的见解,并介绍了一种研究人类-AI协作替代设计的方法。我们将我们的实验平台作为开源提供,以促进未来关于替代设计对人类-AI工作流程影响的研究。
英文摘要:
Details of the designs and mechanisms in support of human-AI collaboration must be considered in the real-world fielding of AI technologies. A critical aspect of interaction design for AI-assisted human decision making are policies about the display and sequencing of AI inferences within larger decision-making workflows. We have a poor understanding of the influences of making AI inferences available before versus after human review of a diagnostic task at hand. We explore the effects of providing AI assistance at the start of a diagnostic session in radiology versus after the radiologist has made a provisional decision. We conducted a user study where 19 veterinary radiologists identified radiographic findings present in patients' X-ray images, with the aid of an AI tool. We employed two workflow configurations to analyze (i) anchoring effects, (ii) human-AI team diagnostic performance and agreement, (iii) time spent and confidence in decision making, and (iv) perceived usefulness of the AI. We found that participants who are asked to register provisional responses in advance of reviewing AI inferences are less likely to agree with the AI regardless of whether the advice is accurate and, in instances of disagreement with the AI, are less likely to seek the second opinion of a colleague. These participants also reported the AI advice to be less useful. Surprisingly, requiring provisional decisions on cases in advance of the display of AI inferences did not lengthen the time participants spent on the task. The study provides generalizable and actionable insights for the deployment of clinical AI tools in human-in-the-loop systems and introduces a methodology for studying alternative designs for human-AI collaboration. We make our experimental platform available as open source to facilitate future research on the influence of alternate designs on human-AI workflows.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。