多模态大语言模型中的对齐漂移：对八个模型版本的双阶段纵向评估
Alignment Drift in Multimodal LLMs: A Two-Phase, Longitudinal Evaluation of Harm Across Eight Model Releases
AI总结：
研究通过双阶段评估揭示多模态大语言模型在不同模型版本中的安全性和对齐漂移问题，强调了持续多模态基准测试的重要性。
摘要：
多模态大语言模型（MLLMs）正越来越多地部署到现实系统中，但其在对抗性提示下的安全性仍缺乏深入研究。我们使用一个固定的基准测试集（726个由26名专业红队成员撰写的对抗性提示）对MLLM的无害性进行了双阶段评估。第一阶段评估了GPT-4o、Claude Sonnet 3.5、Pixtral 12B和Qwen VL Plus；第二阶段评估了它们的后续版本（GPT-5、Claude Sonnet 4.5、Pixtral Large和Qwen Omni），产生了82,256人害评分。在模型家族之间出现了显著且持续差异：Pixtral模型始终是最脆弱的，而Claude模型由于高拒绝率显得最安全。攻击成功率（ASR）显示了明显的对齐漂移：GPT和Claude模型在各代之间表现出攻击成功率的增加，而Pixtral和Qwen则表现出适度的减少。模态效应也随时间变化：在第一阶段，纯文本提示更有效，而第二阶段产生了模型特定的模式，GPT-5和Claude 4.5在不同模态中表现出接近相等的脆弱性。这些发现表明，MLLM的无害性在更新中既不统一也不稳定，强调了需要纵向、多模态基准测试来跟踪不断演变的安全行为。
英文摘要：
Multimodal large language models (MLLMs) are increasingly deployed in real-world systems, yet their safety under adversarial prompting remains underexplored. We present a two-phase evaluation of MLLM harmlessness using a fixed benchmark of 726 adversarial prompts authored by 26 professional red teamers. Phase 1 assessed GPT-4o, Claude Sonnet 3.5, Pixtral 12B, and Qwen VL Plus; Phase 2 evaluated their successors (GPT-5, Claude Sonnet 4.5, Pixtral Large, and Qwen Omni) yielding 82,256 human harm ratings. Large, persistent differences emerged across model families: Pixtral models were consistently the most vulnerable, whereas Claude models appeared safest due to high refusal rates. Attack success rates (ASR) showed clear alignment drift: GPT and Claude models exhibited increased ASR across generations, while Pixtral and Qwen showed modest decreases. Modality effects also shifted over time: text-only prompts were more effective in Phase 1, whereas Phase 2 produced model-specific patterns, with GPT-5 and Claude 4.5 showing near-equivalent vulnerability across modalities. These findings demonstrate that MLLM harmlessness is neither uniform nor stable across updates, underscoring the need for longitudinal, multimodal benchmarks to track evolving safety behaviour.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
