AI 中文总结
本文提出针对红外适配视觉语言模型的InfraPatch定向灰度补丁攻击框架，在10种模型变体上实现86.00%-100%的定向攻击成功率，揭示红外多模态模型的脆弱性，推动其鲁棒性评估。
AI 中文摘要
红外视觉语言模型（IR-VLMs）已成为低能见度条件下多模态感知的有前景范式，但其对定向对抗攻击的鲁棒性仍知之甚少。现有对抗补丁方法主要研究基于RGB的模型或单一下游任务，未探究局部扰动是否能在IR-VLMs中诱导预期语义目标。我们提出InfraPatch，这是一种针对IR-VLMs的白盒、逐实例定向数字灰度补丁攻击框架。InfraPatch在约5%的局部区域预算内优化紧凑的单通道补丁，结合代理引导的放置与任务自适应语义目标，可在图像分类、图像字幕和二元视觉问答中诱导目标行为。我们对10种红外适配模型变体进行评估，使用通过DiffV2IR应用于固定30类COCO子集生成的300张合成红外风格图像，采用干净条件下的定向成功标准。InfraPatch在10种变体上实现了86.00%至100%的定向攻击成功率；在CLIP和BLIP-2上，代理位置搜索相比优化随机放置分别提升了6.67和10.33个百分点的成功率；LLaVA-1.5在两种设置下均保持近100%的饱和状态。补丁区域和目标的消融实验进一步揭示了不同架构和任务格式间存在显著的脆弱性差异。这些结果表明，在受控数字威胁模型下，小型灰度补丁可在各类IR-VLM家族中注入选定的目标语义，为红外多模态系统开展更强的鲁棒性评估提供了动机。
英文摘要
Infrared vision-language models (IR-VLMs) have emerged as a promising paradigm for multimodal perception under low-visibility conditions, yet their robustness to targeted adversarial attacks remains poorly understood. Existing adversarial patch methods mainly study RGB-based models or a single downstream task and do not characterize whether localized perturbations can induce an intended semantic target in IR-VLMs. We propose InfraPatch, a white-box, per-instance framework for targeted digital grayscale patch attacks against IR-VLMs. InfraPatch optimizes a compact single-channel patch within an approximately 5% local-area budget, combines proxy-guided placement with task-adaptive semantic objectives, and induces target behaviors in image classification, image captioning, and binary visual question answering. We evaluate ten infrared-adapted model variants on 300 synthetic infrared-style images generated by applying DiffV2IR to a fixed 30-category COCO subset, using clean-conditioned targeted success criteria. InfraPatch achieves targeted attack success rates from 86.00% to 100% across the ten variants. On CLIP and BLIP-2, proxy location search improves success by 6.67 and 10.33 percentage points over optimized random placement, respectively; LLaVA-1.5 remains saturated near 100% under both settings. Patch-area and objective ablations further expose substantial differences in vulnerability across architectures and task formats. These results show that small grayscale patches can inject chosen target semantics across IR-VLM families under a controlled digital threat model, motivating stronger robustness evaluation for infrared multimodal systems.
