SHOVIR：评估放射学报告生成中视觉捷径学习的基准
SHOVIR: A Benchmark for Evaluating Vision Shortcut Learning in Radiology Report Generation
AI总结：
提出SHOVIR基准，通过遮挡实验检测放射学报告生成模型是否依赖视觉捷径而非真实病理证据，发现高报告质量模型未必具有良好空间定位能力。
摘要：
当前放射学报告生成（RRG）中视觉语言模型（VLM）的评估协议依赖于报告级指标，这些指标衡量词汇重叠或聚合临床正确性。然而，此类指标并不测试单个诊断陈述是否源于图像中可见的实际病理证据。这使得模型能够通过利用学习到的先验或虚假相关性获得竞争性分数，我们将这种失败模式称为视觉捷径。我们引入了SHOVIR，一个用于评估RRG中视觉捷径行为的基准。SHOVIR扩展了两个带有空间标注的胸部X光数据集MIMIC-CXR和PadChest-GR，添加了逐框CheXpert标签，并定义了图像级和疾病级遮挡实验，将干净图像上的基线性能与局部、区域特定扰动进行对比。通过比较这些条件下的预测，我们在疾病类别层面隔离出两种失败模式：直接捷径（在移除视觉证据后发现仍然存在）和上下文捷径（尽管目标区域保持完整，但一旦共病病理被遮挡，检测就会退化）。对八个最先进的VLM进行基准测试，我们发现捷径行为在架构和数据集之间存在显著差异。达到最高基线报告质量的模型并不一定在空间定位上排名最高，揭示出临床流畅的生成可以与对视觉证据的浅层依赖共存。这些发现暴露了当前RRG评估中的盲点，并推动了区域感知评估协议的发展。
英文摘要：
Current evaluation protocols for Vision-Language Models (VLMs) in Radiology Report Generation (RRG) rely on report-level metrics that measure lexical overlap or aggregate clinical correctness. However, such metrics do not test whether individual diagnostic statements stem from the actual pathological evidence visible in the image. This allows models to achieve competitive scores by exploiting learned priors or spurious correlations, a failure mode we refer to as vision shortcut. We introduce SHOVIR, a benchmark for evaluating vision shortcut behavior in RRG. SHOVIR extends two spatially annotated chest X-ray datasets, MIMIC-CXR and PadChest-GR, with per-box CheXpert labels, and defines image-level and disease-level occlusion experiments that contrast baseline performance on clean images against localized, region-specific perturbations. Comparing predictions across these conditions isolates two failure modes at the disease-class level: direct shortcuts, where a finding persists after its visual evidence is removed, and contextual shortcuts, where detection degrades once co-occurring pathologies are occluded despite the target region remaining intact. Benchmarking eight state-of-the-art VLMs, we find that shortcut behavior varies substantially across architectures and datasets. Models achieving the highest baseline report quality do not necessarily rank highest in spatial grounding, revealing that clinically fluent generation can coexist with shallow reliance on visual evidence. These findings expose a blind spot in current RRG evaluation and motivate region-aware assessment protocols.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
