arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.15477cs.CV
收藏

MediConfusion:你能信任你的AI放射科医生吗?探究多模态医疗基础模型的可靠性

MediConfusion: Can you trust your AI radiologist? Probing the reliability of multimodal medical foundation models

Department of Electrical and Computer Engineering, University of Southern California(电气与计算机工程系,南加州大学);Department of Radiology and Imaging Sciences, University of Utah(放射学与影像科学系,犹他大学)

Mohammad Shahab Sepehri, Zalan Fabian, Maryam Soltanolkotabi, Mahdi Soltanolkotabi

AI总结:

本文提出医疗视觉问答基准数据集MediConfusion,从视觉角度探究多模态医疗大语言模型的失效模式,发现现有开源及专有模型表现均低于随机猜测水平,并提取失效模式以助力更可靠的医疗MLLMs设计。

摘要:

多模态大语言模型(MLLMs)通过提供自动化解决方案或作为医疗专业人员的辅助工具,在提升医疗保健的准确性、可及性和成本效益方面具有巨大潜力。尽管过去几年医疗MLLMs的开发取得了初步进展,但人们对其能力和局限性的了解仍不充分。近期,已有多个基准数据集被提出,用于测试这类模型在多个医疗领域的通用医疗知识。然而,这类模型的系统性失效模式和脆弱性仍严重缺乏研究,大多数医疗基准未能揭示现有模型在这一安全关键领域的缺陷。本文提出了MediConfusion,一个具有挑战性的医疗视觉问答(VQA)基准数据集,从视觉角度探究医疗MLLMs的失效模式。我们发现,当前最先进的模型很容易被成对图像混淆,而这些图像在视觉上并不相似,且对医疗专家而言区分度很高。值得注意的是,所有现有模型(无论是开源还是专有模型)在MediConfusion上的表现都低于随机猜测水平,这引发了人们对现有医疗MLLMs部署于医疗保健领域可靠性的严重担忧。我们还提取了模型失效的常见模式,这可能有助于设计新一代更值得信赖、更可靠的医疗保健领域MLLMs。

英文摘要:

Multimodal Large Language Models (MLLMs) have tremendous potential to improve the accuracy, availability, and cost-effectiveness of healthcare by providing automated solutions or serving as aids to medical professionals. Despite promising first steps in developing medical MLLMs in the past few years, their capabilities and limitations are not well-understood. Recently, many benchmark datasets have been proposed that test the general medical knowledge of such models across a variety of medical areas. However, the systematic failure modes and vulnerabilities of such models are severely underexplored with most medical benchmarks failing to expose the shortcomings of existing models in this safety-critical domain. In this paper, we introduce MediConfusion, a challenging medical Visual Question Answering (VQA) benchmark dataset, that probes the failure modes of medical MLLMs from a vision perspective. We reveal that state-of-the-art models are easily confused by image pairs that are otherwise visually dissimilar and clearly distinct for medical experts. Strikingly, all available models (open-source or proprietary) achieve performance below random guessing on MediConfusion, raising serious concerns about the reliability of existing medical MLLMs for healthcare deployment. We also extract common patterns of model failure that may help the design of a new generation of more trustworthy and reliable MLLMs in healthcare.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑