When Safe Concepts Become Unsafe: Multi-Concept Compositional Vulnerabilities in Text-to-Image Models
School of Cyber Science and Engineering, Xi'an Jiaotong University(西安交通大学计算机科学与工程学院);CISPA Helmholtz Center for Information Security(信息安全研究中心);School of Cyber Science and Engineering, Wuhan University(武汉大学计算机科学与工程学院)
Chaoshuo Zhang, Yibo Liang, Mengke Tian, Chenhao Lin, Zhengyu Zhao, Le Yang, Chong Zhang, Yang Zhang, Qian Wang, Chao Shen
Text-to-image (T2I) models are increasingly optimized for following user instructions faithfully. However, we find that this capability introduces a safety vulnerability we call Multi-Concept Compositional Unsafety (MCCU). MCCU occurs when multiple individually safe concepts, if combined in a single generation request, lead to harmful or sensitive visual outputs. Unlike prior jailbreak settings, MCCU does not rely on adversarial prompts, model access, or explicitly disallowed content. Instead, the risk emerges from how the model composes multiple safe visual concepts into a single scene. To systematically measure this threat, we build TwoHamsters, a large-scale evaluation framework consisting of 20k prompts, 51 curated concept pairs, and six risk categories. We evaluate 13 T2I models under a black-box setting. Our results show a clear conflict between instruction-following and safety: models that follow prompts more faithfully tend to produce more MCCU failures. For example, FLUX.1 achieves a 99.35% Unsafe Alignment Rate while only reaching a 1.57% MCCU Defense Rate. We further evaluate three representative defenses, including safety filtering, MCCU-specific detector fine-tuning, and concept erasure, all of which fail against unseen concept combinations. Our findings suggest that compositional reasoning in T2I models creates an attack surface that is not captured by existing safety mechanisms. We anticipate the release of TwoHamsters will catalyze community development of advanced generative defense mechanisms.
AutoPrompt: Automated Red-Teaming of Text-to-Image Models via LLM-Driven Adversarial Prompts
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络安全学院);School of Cyberspace, Hangzhou Dianzi University(杭州电子科技大学网络空间学院);Nanyang Technological University(南洋理工大学)
Agent-Fence: Mapping Security Vulnerabilities Across Deep Research Agents
Computer Science Department, Southern Illinois University(索尔坦大学计算机科学系);Computer Science Department, University of Texas(德克萨斯大学计算机科学系);Hanyang University(翰阳大学);The Grainger College of Engineering, Nuclear, Plasma & Radiological Engineering, University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校格雷格尔工程学院)
Jailbreaking on Text-to-Video Models via Scene Splitting Strategy
Yonsei University(延世大学);Korea Institute of Science and Technology(韩国科学技术院);AIM Intelligence(AIM智能);Seoul National University(首尔国立大学);Kyung Hee University(庆熙大学)
Detecting speculative leaks with compositional semantics
CISPA Helmholtz Center for Information Security(CISPA赫尔姆霍兹信息安全中心);University of Trento(特伦托大学);IMDEA Software Institute(IMDEA软件研究所);Azure Research, Microsoft(微软Azure研究);Saarland University(萨尔兰大学);Amazon(亚马逊)
MalURLBench: A Benchmark Evaluating Agents' Vulnerabilities When Processing Web URLs
Zhejiang University(浙江大学);National University of Malaysia (Universiti Kebangsaan Malaysia)(马来西亚国立大学);Hangzhou City University(杭州市大学);Yangzhou University(扬州大学);Chongqing University(重庆大学);Binjiang Institute of Zhejiang University(浙江大学滨江研究院)
Secure Tug-of-War (SecTOW): Iterative Defense-Attack Training with Reinforcement Learning for Multimodal Model Security
Institute of Artificial Intelligence (TeleAI), China Telecom, China(人工智能研究院(TeleAI),中国电信,中国);Northwestern Polytechnical University(西北工业大学);China Telecom, China(中国电信,中国)
When Safe Skills Collide: Measuring Compositional Risk in Agent Skill Ecosystems
Carnegie Mellon University(卡内基梅隆大学);Georgia Institute of Technology(佐治亚理工学院);University of Glasgow(格拉斯哥大学);Independent Researcher(独立研究员);Corespeed Inc.(Corespeed公司)
RedEdit: Agentic Red-Teaming of Image Safety Classifiers via MCTS-Guided Photo-Editing
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Weixin AI, Tencent Inc.(腾讯微信AI);School of Science, Sun Yat-sen University(中山大学理学院)
SafePTR: Token-Level Jailbreak Defense in Multimodal LLMs via Prune-then-Restore Mechanism
Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China(电子科技大学深圳研究院);Southwestern University of Finance and Economics(西南财经大学);Engineering Research Center of Intelligent Finance, Ministry of Education(教育部智能金融工程研究中心);Tongji University(同济大学)
DARWIN: Evolving Jailbreak Adversary and Guardrail for LLM Safety Evaluation and Protection
The State Key Laboratory of Blockchain and Data Security, Zhejiang University(浙江大学区块链与数据安全全国重点实验室);Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security(杭州高新技术开发区(滨江)区块链与数据安全研究所);Shanghai AI Laboratory(上海人工智能实验室);East China Normal University(华东师范大学)
HauntAttack: When Attack Follows Reasoning as a Shadow
State Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University(北京大学计算机学院多媒体信息处理国家重点实验室);StepFun;Department of Computing, The Hong Kong Polytechnic University(香港理工大学计算机系);Institute of Artificial Intelligence, Beihang University(北航人工智能研究院)