RedDiffuser：通过强化扩散审计多模态安全故障
RedDiffuser: Auditing Multimodal Safety Failures in Vision-Language Models via Reinforced Diffusion
AI总结：
研究多模态系统在有害上下文暴露下的安全审计，提出RedDiffuser框架通过扩散模型生成视觉输入，揭示隐藏的安全漏洞，实验显示VLMs在部分有毒文本与视觉上下文结合时存在广泛安全问题。
摘要：
大型视觉-语言模型（VLMs）正被越来越多地部署在开放环境中，确保在多模态输入下的可靠性至关重要。然而，现有评估仍以指令为中心，关注显式的恶意查询，而忽视了更现实且未被探索的风险：在有害上下文暴露下，安全对齐是否仍稳健。本工作研究了在有害上下文暴露下的多模态安全审计，探讨VLMs在部分有毒文本与视觉上下文结合时能否保持安全行为。为实现系统审计，我们提出了RedDiffuser（RedDiff），一种基于强化学习的框架，利用扩散模型生成语义连贯的视觉输入进行黑盒安全测试。通过结合贪婪提示搜索与强化优化，RedDiffuser揭示了高风险的多模态输入，暴露了潜在的安全故障。在开源和商业VLMs上的广泛实验显示，此类上下文条件故障普遍存在。在LLaVA上，RedDiffuser在原始数据集上将不安全响应率提高至10.69%，在保留集上提高至8.91%，且能有效迁移到Gemini和LLaMA-Vision。这些漏洞即使在外部安全防护下仍存在，表明当前系统级安全机制仍不足以应对现实中的多模态风险。我们的发现揭示了现有安全评估中的关键盲点，并确立了上下文感知的多模态审计作为诊断现代VLM系统隐藏漏洞的重要范式。
英文摘要：
Large Vision-Language Models (VLMs) are increasingly deployed in open-ended environments, where ensuring reliable safety under multimodal inputs is critical. However, existing evaluations remain largely instruction-centric, focusing on explicit malicious queries while overlooking a more realistic and underexplored risk: whether safety alignment remains robust under harmful contextual exposure. This limitation is particularly important for multimodal systems, where visual inputs can substantially steer model behavior and render text-only auditing insufficient. In this work, we study multimodal safety auditing under harmful contextual exposure, asking whether VLMs can maintain safe behavior when partial toxic text is paired with visual context. To enable systematic auditing, we propose RedDiffuser (RedDiff), a reinforcement-based framework that leverages diffusion models to generate semantically coherent visual inputs for black-box safety testing. By combining greedy prompt search with reinforcement optimization, RedDiffuser uncovers high-risk multimodal inputs that expose latent safety failures. Extensive experiments on both open-source and commercial VLMs show that such context-conditioned failures are widespread. On LLaVA, RedDiffuser increases unsafe response rates by up to 10.69% on the original set and 8.91% on a hold-out set, with strong transferability to Gemini and LLaMA-Vision. These vulnerabilities persist even under external safety guardrails, suggesting that current system-level safety mechanisms remain insufficient for realistic multimodal risks. Our findings reveal a critical blind spot in existing safety evaluations and establish context-aware multimodal auditing as an essential paradigm for diagnosing hidden vulnerabilities in modern VLM systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
