超越视觉取证：审计多模态鲁棒性用于合成医学图像检测
Text Over Image: Auditing Multimodal Robustness in Synthetic Medical Image Detection
AI总结：
针对合成医学图像检测中多模态鲁棒性不足的问题，提出图像-记录配对基准，揭示视觉语言模型因过度依赖记录上下文而导致的预测偏差。
摘要：
随着生成式AI的快速采用，合成医学图像带来了日益增长的风险，包括诊断欺骗和保险欺诈。尽管先前的工作探索了基于视觉语言模型（VLM）的合成图像检测，但这些评估通常孤立地考虑图像。然而，在临床实践中，图像与结构化记录和元数据一起被解读，并且VLM越来越多地在联合图像-记录输入下部署。我们发现了一个先前未被充分研究的多模态漏洞：当同时提供两种模态时，VLM可能在真实性判断中过度加权记录上下文，使得同一张图像仅因伴随文本的变化而得到不同的预测。这引发了在实际部署中鲁棒性的担忧。为了系统地表征这一效应，我们将合成医学图像检测重新定义为图像-记录接口处的多模态鲁棒性审计，并引入了一个配对基准，该基准在保持图像固定的同时交换受控的元数据变体。跨多种成像模态，我们评估了多种开放权重和前沿API VLM，并量化了元数据单独如何改变真实性预测。我们的基准提供了一个标准化工具，用于评估和改进超越纯图像设置的多模态鲁棒性。代码可在该 https URL 获取。
英文摘要：
With the rapid adoption of generative AI, synthetic medical images pose growing risks, including diagnostic deception and insurance fraud. Although prior work has explored vision-language model (VLM)-based synthetic image detection, these evaluations typically consider images in isolation. In clinical practice, however, images are interpreted alongside structured records and metadata, and VLMs are increasingly deployed under joint image-record inputs. We uncover a previously underexamined multimodal vulnerability: when given both modalities, VLMs may overweight record context in authenticity judgments, such that the same image receives different predictions solely due to changes in its accompanying text. This raises concerns about robustness in real-world deployment. To systematically characterize this effect, we reformulate synthetic medical image detection as an audit of multimodal robustness at the image-record interface and introduce a paired benchmark that holds the image fixed while swapping controlled metadata variants. Across multiple imaging modalities, we evaluate diverse open-weight and frontier API VLMs and find that changing the metadata context alone can flip authenticity judgments, with accuracy on authentic images dropping by 61.1% on average under an explicit AI-origin tag. We further propose an inference-time mitigation pipeline that detects and neutralizes provenance shortcuts without model retraining, substantially outperforming direct prompt-based suppression on the affected subset. Our benchmark provides a standardized tool for assessing and improving multimodal robustness beyond image-only settings. Code and data will be released upon acceptance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
