Mosaic: 通过多视图集成优化实现对闭源视觉语言模型的多模态劫持
Mosaic: Multimodal Jailbreak against Closed-Source VLMs via Multi-View Ensemble Optimization
AI总结：
本文提出Mosaic框架，通过多视图集成优化减少对单一代理模型的依赖，提升对闭源VLMs的多模态劫持效果，实验显示其在安全基准上的攻击成功率和毒性均达到最优。
摘要：
视觉语言模型（VLMs）虽然强大，但仍易受多模态劫持攻击。现有攻击主要依赖于显式视觉提示攻击或基于梯度的对抗优化。前者更容易被检测到，后者会产生微妙的扰动，更加不易察觉，但通常在同质的开源替代目标设置下进行优化和评估，使其在商业闭源VLMs上的有效性在异质设置下仍不明确。为探讨这一问题，我们研究了不同的替代目标设置，并观察到同质与异质设置之间存在一致的差距，这种现象我们称为替代依赖性。受这一发现的启发，我们提出了Mosaic，一种用于多模态劫持的多视图集成优化框架，旨在对抗闭源VLMs。Mosaic通过减少对任何单一替代模型和视觉视图的过度依赖，缓解了异质替代目标设置下的替代依赖性。具体而言，Mosaic包含三个核心组件：一个文本侧转换模块，扰动敏感的拒绝性词法模式；一个多视图图像优化模块，更新扰动以避免过度拟合单一视觉视图；以及一个替代模型引导模块，聚合多个替代VLMs的优化信号以减少替代特定偏差。在安全基准上的大量实验表明，Mosaic在商业闭源VLMs上实现了最先进的攻击成功率和平均毒性。
英文摘要：
Vision-Language Models (VLMs) are powerful but remain vulnerable to multimodal jailbreak attacks. Existing attacks mainly rely on either explicit visual prompt attacks or gradient-based adversarial optimization. While the former is easier to detect, the latter produces subtle perturbations that are less perceptible, but is usually optimized and evaluated under homogeneous open-source surrogate-target settings, leaving its effectiveness on commercial closed-source VLMs under heterogeneous settings unclear. To examine this issue, we study different surrogate-target settings and observe a consistent gap between homogeneous and heterogeneous settings, a phenomenon we term surrogate dependency. Motivated by this finding, we propose Mosaic, a Multi-view ensemble optimization framework for multimodal jailbreak against closed-source VLMs, which alleviates surrogate dependency under heterogeneous surrogate-target settings by reducing over-reliance on any single surrogate model and visual view. Specifically, Mosaic incorporates three core components: a Text-Side Transformation module, which perturbs refusal-sensitive lexical patterns; a Multi-View Image Optimization module, which updates perturbations under diverse cropped views to avoid overfitting to a single visual view; and a Surrogate Ensemble Guidance module, which aggregates optimization signals from multiple surrogate VLMs to reduce surrogate-specific bias. Extensive experiments on safety benchmarks demonstrate that Mosaic achieves state-of-the-art Attack Success Rate and Average Toxicity against commercial closed-source VLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
