arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2502.03482eess.IVcs.AIcs.CVcs.CYcs.HCcs.LG

领域专家能否恰当地依赖AI?一项关于AI辅助前列腺癌MRI诊断的案例研究

Can Domain Experts Rely on AI Appropriately? A Case Study on AI-Assisted Prostate Cancer MRI Diagnosis

University of Chicago(芝加哥大学);Toyota Technological Institute at Chicago(芝加哥丰田技术研究所);University of Michigan(密歇根大学);Bagcilar Training and Research Hospital(巴格奇拉尔培训与研究医院);Hospital of Bolzano (SABES-ASDAA)(博尔扎诺医院(SABES-ASDAA));Radiology Associates of North Texas(北德克萨斯放射协会);İstanbul Medipol University Hospital(伊斯坦布尔梅迪波尔大学医院)
Chacha Chen, Han Liu, Jiamin Yang, Benjamin M. Mervak, Bora Kalaycioglu, Grace Lee, Emre Cakmakli, Matteo Bonatti, Sridhar Pudu, Osman Kahraman, Gul Gizem Pamuk… 展开作者

Chacha Chen, Han Liu, Jiamin Yang, Benjamin M. Mervak, Bora Kalaycioglu, Grace Lee, Emre Cakmakli, Matteo Bonatti, Sridhar Pudu, Osman Kahraman, Gul Gizem Pamuk, Aytekin Oto, Aritrick Chatterjee, Chenhao Tan

AI总结:

本文通过与放射科医生合作开展两项实验,研究AI辅助和性能反馈对前列腺癌MRI诊断决策的影响,发现人类-AI团队优于单独人类但不及单独AI,且人类-AI团队集成可超越单独AI,为优化人机协作提供了方向。

摘要:

尽管人们对人类与AI共同决策的兴趣日益增长,但涉及领域专家的实验研究仍然很少,这主要是由于与领域专家合作的复杂性以及建立现实实验环境的挑战。在这项工作中,我们与放射科医生在前列腺癌MRI图像诊断方面开展了深入合作。基于现有的前列腺癌诊断教学工具,我们开发了一个界面,并进行了两项实验,以研究AI辅助和性能反馈如何影响领域专家的决策。在研究1中,临床医生被要求先给出初步诊断(人类),然后查看AI的预测,最后确定他们的最终决策(人类-AI团队)。在研究2中(经过一段记忆消退期后),相同的参与者首先收到来自研究1的汇总性能统计数据,具体包括他们自己的表现、AI的表现以及他们的人类-AI团队表现,然后直接查看AI的预测,再做出诊断(即没有独立的初步诊断)。这两种工作流程代表了临床AI工具在实践中可能被使用的现实方式,其中第二项研究模拟了医生可以根据先前的性能反馈调整他们对AI的依赖和信任的场景。我们的研究结果表明,虽然人类-AI团队始终优于单独的人类,但由于依赖不足,它们仍然不如AI,这与之前使用众包工作者的研究类似。向临床医生提供性能反馈并没有显著提高人类-AI团队的表现,尽管提前展示AI决策会促使人们更多地遵循AI。同时,我们观察到人类-AI团队的集成可以优于单独的AI,这为人类-AI协作指明了有前景的方向。

英文摘要:

Despite the growing interest in human-AI decision making, experimental studies with domain experts remain rare, largely due to the complexity of working with domain experts and the challenges in setting up realistic experiments. In this work, we conduct an in-depth collaboration with radiologists in prostate cancer diagnosis based on MRI images. Building on existing tools for teaching prostate cancer diagnosis, we develop an interface and conduct two experiments to study how AI assistance and performance feedback shape the decision making of domain experts. In Study 1, clinicians were asked to provide an initial diagnosis (human), then view the AI's prediction, and subsequently finalize their decision (human-AI team). In Study 2 (after a memory wash-out period), the same participants first received aggregated performance statistics from Study 1, specifically their own performance, the AI's performance, and their human-AI team performance, and then directly viewed the AI's prediction before making their diagnosis (i.e., no independent initial diagnosis). These two workflows represent realistic ways that clinical AI tools might be used in practice, where the second study simulates a scenario where doctors can adjust their reliance and trust on AI based on prior performance feedback. Our findings show that, while human-AI teams consistently outperform humans alone, they still underperform the AI due to under-reliance, similar to prior studies with crowdworkers. Providing clinicians with performance feedback did not significantly improve the performance of human-AI teams, although showing AI decisions in advance nudges people to follow AI more. Meanwhile, we observe that the ensemble of human-AI teams can outperform AI alone, suggesting promising directions for human-AI collaboration.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑