揭示物理世界语义漏洞：用于红外视觉语言模型的通用对抗性补丁
Revealing Physical-World Semantic Vulnerabilities: Universal Adversarial Patch for Infrared Vision-Language Models
AI总结：
本文提出通用曲网格补丁框架UCGP，用于揭示红外视觉语言模型的语义鲁棒性漏洞，通过扰动视觉表征空间以削弱跨模态语义对齐，验证了其在多种架构和数据集上的有效性。
摘要：
红外视觉语言模型（IR-VLMs）在低能见度环境中展现出多模态感知的潜力，但其对对抗攻击的鲁棒性仍缺乏研究。现有对抗补丁方法主要用于RGB模型的封闭集场景，不适用于红外VLMs的开放语义理解和物理部署需求。为此，我们提出通用曲网格补丁（UCGP）框架，整合曲网格网格参数化以生成连续、低频且可部署的补丁，并通过统一的表征驱动目标促进子空间脱离、拓扑破坏和隐蔽性。为提高现实部署和领域转移下的鲁棒性，我们进一步结合元差分进化和EOT增强的TPS变形建模。不同于操纵标签或提示，UCGP直接破坏视觉表征空间，削弱跨模态语义对齐。大量实验表明，UCGP在多种红外VLM架构上一致削弱语义理解，同时保持跨模型可转移性、跨数据集泛化性、现实物理有效性及对防御的鲁棒性。这些发现揭示了当前红外多模态系统中被忽视的鲁棒性漏洞。
英文摘要：
Infrared vision-language models (IR-VLMs) are becoming important for semantic perception in low-visibility environments, yet their robustness to physical semantic attacks remains underexplored. Existing adversarial patch methods are mainly designed for red-green-blue (RGB) images or closed-set infrared detectors and do not directly address open-ended IR-VLM tasks, where one deployable artifact can affect classification, captioning, and visual question answering (VQA) simultaneously. We propose Universal Curved-Grid Patch, abbreviated UCGP, a universal physical adversarial patch framework tailored to IR-VLMs. UCGP represents the patch as a deployable low-frequency curved grid and optimizes it with a representation-driven objective over subspace departure, topology disruption, and local appearance regularization. Meta Differential Evolution (MetaDE) searches for patch parameters with two physical robustness augmentations: Expectation over Transformation (EOT) capture sampling for imaging variations and thin-plate spline (TPS) patch-deformation modeling for non-rigid local shape changes. Rather than manipulating labels or prompts, UCGP disrupts the clean-category manifold in visual representation space, which later appears as degraded cross-modal outputs. Experiments show that a single shared patch degrades classification, captioning, and VQA across diverse IR-VLM architectures while retaining measurable cross-model transfer, cross-dataset generalization, cross-category extensibility, and real-scene physical effectiveness under the evaluated conditions. These results reveal a robustness blind spot in current infrared multimodal systems. Code and reproducibility assets are available at https://github.com/dyx6663/UCGP.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
