视觉-语言模型在对抗性频域扰动下的可靠性
On the Reliability of Vision-Language Models Under Adversarial Frequency-Domain Perturbations
AI总结：
本文揭示了视觉-语言模型在对抗性频域扰动下的脆弱性，通过设计频域图像变换方法证明该扰动能破坏真实性检测和图像描述任务，并跨多个模型与数据集验证了其可靠性问题。
摘要：
视觉-语言模型（VLMs）越来越多地被用作视觉内容推理的感知模块，包括通过图像描述和DeepFake检测。在这项工作中，我们揭示了VLMs在暴露于频域中微妙、结构化扰动时的关键脆弱性。具体而言，我们强调了这些特征变换如何破坏真实性/DeepFake检测和自动图像描述任务。我们设计了针对性的图像变换，在频域中操作，以在暴露于频率扰动的真实和合成图像时系统地调整VLM输出。我们证明，这种扰动注入方法可泛化至五个最先进的VLMs，包括不同参数的Qwen2/2.5和BLIP模型。在十个真实和生成的图像数据集上的实验表明，VLM判断对基于频率的线索敏感，且可能不完全与语义内容一致。至关重要的是，我们展示了视觉上不可感知的空间频率变换暴露了部署用于自动图像描述和真实性检测任务的VLMs的脆弱性。我们在现实、黑盒约束下的发现挑战了VLMs的可靠性，强调了构建鲁棒多模态感知系统的必要性。
英文摘要：
Vision-Language Models (VLMs) are increasingly used as perceptual modules for visual content reasoning, including through captioning and DeepFake detection. In this work, we expose a critical vulnerability of VLMs when exposed to subtle, structured perturbations in the frequency domain. Specifically, we highlight how these feature transformations undermine authenticity/DeepFake detection and automated image captioning tasks. We design targeted image transformations, operating in the frequency domain to systematically adjust VLM outputs when exposed to frequency-perturbed real and synthetic images. We demonstrate that the perturbation injection method generalizes across five state-of-the-art VLMs which includes different-parameter Qwen2/2.5 and BLIP models. Experimenting across ten real and generated image datasets reveals that VLM judgments are sensitive to frequency-based cues and may not wholly align with semantic content. Crucially, we show that visually-imperceptible spatial frequency transformations expose the fragility of VLMs deployed for automated image captioning and authenticity detection tasks. Our findings under realistic, black-box constraints challenge the reliability of VLMs, underscoring the need for robust multimodal perception systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
