arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.15967cs.CRcs.CV
收藏

TwoHamsters:文本到图像模型中多概念组合不安全性的基准测试

When Safe Concepts Become Unsafe: Multi-Concept Compositional Vulnerabilities in Text-to-Image Models

School of Cyber Science and Engineering, Xi'an Jiaotong University(西安交通大学计算机科学与工程学院);CISPA Helmholtz Center for Information Security(信息安全研究中心);School of Cyber Science and Engineering, Wuhan University(武汉大学计算机科学与工程学院)

Chaoshuo Zhang, Yibo Liang, Mengke Tian, Chenhao Lin, Zhengyu Zhao, Le Yang, Chong Zhang, Yang Zhang, Qian Wang, Chao Shen

AI总结:

本文提出TwoHamsters基准,通过17500个提示测试文本到图像模型在多概念组合不安全性的表现,揭示现有模型和防御机制在处理危险组合生成时的局限性。

摘要:

尽管文本到图像(T2I)模型展现出显著的合成能力,但确保其免受内容违规仍是一个持续的挑战。现有的安全对齐主要关注显而易见的恶意概念,往往忽视了组合语义中微妙但关键的风险。为了解决这一遗漏,我们识别并正式化了一种新的漏洞:多概念组合不安全性(MCCU),其中不安全的语义源于个别良性概念的隐含关联。基于这一公式,我们引入了TwoHamsters,一个包含17500个提示的全面基准,旨在探测MCCU漏洞。通过严格评估10种最先进的模型和16种防御机制,我们的分析得出8项关键见解。特别是,我们证明当前T2I模型和防御机制面临严重的MCCU风险:在TwoHamsters上,FLUX的MCCU生成成功率为99.52%,而LLaVA-Guard仅达到41.06%的召回率,突显了当前范式在管理危险组合生成方面的关键限制。

英文摘要:

Text-to-image (T2I) models are increasingly optimized for following user instructions faithfully. However, we find that this capability introduces a safety vulnerability we call Multi-Concept Compositional Unsafety (MCCU). MCCU occurs when multiple individually safe concepts, if combined in a single generation request, lead to harmful or sensitive visual outputs. Unlike prior jailbreak settings, MCCU does not rely on adversarial prompts, model access, or explicitly disallowed content. Instead, the risk emerges from how the model composes multiple safe visual concepts into a single scene. To systematically measure this threat, we build TwoHamsters, a large-scale evaluation framework consisting of 20k prompts, 51 curated concept pairs, and six risk categories. We evaluate 13 T2I models under a black-box setting. Our results show a clear conflict between instruction-following and safety: models that follow prompts more faithfully tend to produce more MCCU failures. For example, FLUX.1 achieves a 99.35% Unsafe Alignment Rate while only reaching a 1.57% MCCU Defense Rate. We further evaluate three representative defenses, including safety filtering, MCCU-specific detector fine-tuning, and concept erasure, all of which fail against unseen concept combinations. Our findings suggest that compositional reasoning in T2I models creates an attack surface that is not captured by existing safety mechanisms. We anticipate the release of TwoHamsters will catalyze community development of advanced generative defense mechanisms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑