arXiv 2605.171402026-05-19cs.CVcs.AIcs.CL

UCSF-PDGM-VQA: 用于脑肿瘤MRI解读的视觉问答数据集

UCSF-PDGM-VQA: Visual Question Answering dataset for brain tumor MRI interpretation

Fung Institute for Engineering Leadership（工程领导力基金会）；University of California, Berkeley（加州大学伯克利分校）；Department of Radiology（放射科）；University of California, San Francisco（加州大学旧金山分校）；Division of Clinical Informatics and Digital Transformation（临床信息学与数字转型部）；Department of Neurological Surgery（神经外科部）

Shiv Ghosh, Junayd Lateef, Chih-Hua Liu, Yannan Yu, Andreas M. Rauschecker, Madhumita Sushil

备注：: 10 pages, 2 figures, 6 tables

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出一个临床相关的视觉问答基准数据集UCSF-PDGM-VQA，包含2387个问题-答案对，用于评估视觉语言模型在处理多序列3D MRI扫描中的能力，发现现有模型在多模态处理上存在缺陷。

摘要：

脑肿瘤诊断很大程度上依赖于磁共振成像（MRI）评估，这需要放射科医生综合分析成千上万张来自多种3D序列和纵向研究的图像。这一过程需要高级的神经放射学培训，具有显著的认知负荷，并且非常耗时。尽管放射学需求不断增长，但这种专业知识难以扩展，给当前的医疗系统带来压力。视觉-语言模型（VLMs）提供了一种通过半自动化、互动解释复杂脑MRI来减轻这种负担的机会。然而，由于缺乏专门的评估基准，它们在神经肿瘤学中目前使用有限。我们介绍了一个临床相关的视觉问答（VQA）基准——UCSF-PDGM-VQA数据集，包含来自公共UCSF-PDGM数据集中473个胶质瘤相关MRI研究的2387个QA对。我们进一步在该数据集上建立了六种最先进的视觉语言模型（VLMs）和一个大型语言模型的性能基线。我们发现，当前模型无法有效处理多序列、三维MRI扫描，导致视觉特征的抑制和对语言先验的过度依赖，从而造成模态崩溃。这些发现突显了当前模型在临床环境中的可靠性和安全性方面的关键缺陷，需要开发稳健的、领域特定的VLMs。

英文摘要：

Brain tumor diagnosis is largely dependent on Magnetic Resonance Imaging (MRI) evaluation, which requires radiologists to synthesize thousands of images across multiple 3D sequences and longitudinal studies. This process requires advanced neuro-radiology training, poses substantial cognitive load, and is highly time-consuming. Despite increasing demands in radiology, this expertise is difficult to scale, straining the current health systems. Vision-Language Models (VLMs) provide an opportunity to reduce this burden through a semi-automated, interactive interpretation of complex brain MRIs. However, they are currently underutilized in neuro-oncology due to a lack of specialized benchmarks for evaluating them. We introduce a clinically relevant visual question answering (VQA) benchmark -- the UCSF-PDGM-VQA dataset -- consisting of 2,387 QA pairs from 473 glioma-related MRI studies in the public UCSF-PDGM dataset. We further establish a performance baseline for six state-of-the-art vision-language models (VLMs) and one large language model on this dataset. We find that current models are incapable of effectively processing multi-sequence, 3-dimensional MRI scans, thus resulting in a suppression of visual features and over-reliance on language priors, causing modality collapse. These findings underscore a critical deficiency in current model reliability and safety within clinical settings, necessitating the development of robust, domain-specific VLMs.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。