视觉语言模型对分裂图像有害输入攻击的鲁棒性
Robustness of Vision Language Models Against Split-Image Harmful Input Attacks
AI总结：
本研究提出分裂图像视觉陷阱攻击（SIVA），揭示视觉语言模型在面对分裂图像攻击时的安全漏洞，并通过对抗性知识蒸馏算法提升跨模型攻击效果。
摘要：
视觉语言模型（VLMs）现在是现代AI的核心组成部分。近期的研究提出了几种使用单张或整体图像的视觉陷阱攻击。然而，由于通过偏好优化（例如RLHF）进行广泛的安全部署，当前的VLMs在对抗这些攻击时表现出较强的鲁棒性。在本工作中，我们识别出一个新的漏洞：虽然VLM预训练和指令微调能够很好地推广到分裂图像输入，但安全对齐通常只在整体图像上进行，并不考虑分布在多个图像碎片中的有害语义。因此，VLMs往往无法检测并拒绝有害的分裂图像输入，其中不安全的线索只有在组合图像后才会出现。我们引入了新的分裂图像视觉陷阱攻击（SIVA），利用这种不一致。与先前基于优化的攻击不同，由于模型架构和先验不匹配，这些攻击在黑盒转移性方面表现较差。我们的攻击在逐步的阶段中从简单的分裂发展到适应性的白盒攻击，最终形成黑盒转移攻击。我们的最强策略利用了新的对抗性知识蒸馏（Adv-KD）算法，显著提高了跨模型的转移能力。在三个最先进的现代VLMs和三个陷阱数据集上的评估显示，我们的最强攻击在现有基准上实现了高达60%的转移成功率提升。最后，我们提出了有效的方法来解决当前VLM安全对齐中的这一关键漏洞。
英文摘要：
Vision-Language Models (VLMs) are now a core part of modern AI. Recent work proposed several visual jailbreak attacks using single/ holistic images. However, contemporary VLMs demonstrate strong robustness against such attacks due to extensive safety alignment through preference optimization, e.g., reinforcement learning from human feedback (RLHF). In this work, we identify a new vulnerability: while VLM pretraining and instruction tuning generalize well to split-image inputs, safety alignment is typically performed only on holistic images and does not account for harmful semantics distributed across multiple image fragments. Consequently, VLMs often fail to detect and reject harmful split-image inputs, in which unsafe cues emerge only upon combining images. We introduce novel split-image visual jailbreak attacks (\textbf{SIVA}) that exploit this misalignment. Unlike prior optimization-based attacks, which exhibit poor black-box transferability due to architectural and prior mismatches across models, our attacks evolve in progressive phases from naive splitting to an adaptive white-box attack, culminating in a black-box transfer attack. Our strongest strategy leverages a novel adversarial knowledge distillation \textbf{(Adv-KD)} algorithm to substantially improve cross-model transferability. Evaluations on four state-of-the-art modern VLMs and three jailbreak datasets demonstrate that our strongest attack achieves up to 44% higher transfer success than existing baselines. Lastly, we propose efficient ways to address this critical vulnerability in the current VLM safety alignment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
