用可部署的多模态语义照明攻击挑战视觉-语言模型
Challenging Vision-Language Models with Physically Deployable Multimodal Semantic Lighting Attacks
AI总结：
本文提出MSLA攻击框架，通过可控对抗性照明攻击视觉-语言模型的多模态语义理解，揭示其在现实世界中的安全风险。
摘要：
视觉-语言模型（VLMs）表现出色，但其安全性仍不明确。现有对抗研究几乎只关注数字环境，忽略了物理世界威胁。随着VLMs在现实环境中的部署增加，这种差距变得关键，因为对抗扰动必须物理可实现。尽管有实际相关性，针对VLMs的物理攻击尚未系统研究。此类攻击可能引起识别失败并进一步破坏多模态推理，导致下游任务的严重语义误解。因此，研究VLMs的物理攻击对评估其现实安全风险至关重要。为填补这一空白，我们提出了多模态语义照明攻击（MSLA），这是首个可部署的对抗攻击框架。MSLA利用可控对抗性照明在现实场景中破坏多模态语义理解，攻击语义对齐而非仅任务特定输出。因此，它会降低主流CLIP变体的零样本分类性能，并在图像描述和视觉问答（VQA）中引起严重语义幻觉。在数字和物理领域进行的广泛实验表明，MSLA有效、可转移且实际可实现。我们的发现提供了首个证据，表明VLMs对可部署的语义攻击高度易感，揭示了一个之前被忽视的鲁棒性差距，并强调了对VLMs现实世界鲁棒性评估的紧迫需求。
英文摘要：
Vision-Language Models (VLMs) have shown remarkable performance, yet their security remains insufficiently understood. Existing adversarial studies focus almost exclusively on the digital setting, leaving physical-world threats largely unexplored. As VLMs are increasingly deployed in real environments, this gap becomes critical, since adversarial perturbations must be physically realizable. Despite this practical relevance, physical attacks against VLMs have not been systematically studied. Such attacks may induce recognition failures and further disrupt multimodal reasoning, leading to severe semantic misinterpretation in downstream tasks. Therefore, investigating physical attacks on VLMs is essential for assessing their real-world security risks. To address this gap, we propose Multimodal Semantic Lighting Attacks (MSLA), the first physically deployable adversarial attack framework against VLMs. MSLA uses controllable adversarial lighting to disrupt multimodal semantic understanding in real scenes, attacking semantic alignment rather than only task-specific outputs. Consequently, it degrades zero-shot classification performance of mainstream CLIP variants while inducing severe semantic hallucinations in advanced VLMs such as LLaVA and BLIP across image captioning and visual question answering (VQA). Extensive experiments in both digital and physical domains demonstrate that MSLA is effective, transferable, and practically realizable. Our findings provide the first evidence that VLMs are highly vulnerable to physically deployable semantic attacks, exposing a previously overlooked robustness gap and underscoring the urgent need for physical-world robustness evaluation of VLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
