Augmenting language models with image inputs may enable more effective jailbreak attacks through continuous optimization, unlike text inputs that require discrete optimization. However, new multimodal fusion models tokenize all input modalities using non-differentiable functions, which hinders straightforward attacks. In this work, we introduce the notion of a tokenizer shortcut that approximates tokenization with a continuous function and enables continuous optimization. We use tokenizer shortcuts to create the first end-to-end gradient image attacks against multimodal fusion models. We evaluate our attacks on Chameleon models and obtain jailbreak images that elicit harmful information for 72.5% of prompts. Jailbreak images outperform text jailbreaks optimized with the same objective and require 3x lower compute budget to optimize 50x more input tokens. Finally, we find that representation engineering defenses, like Circuit Breakers, trained only on text attacks can effectively transfer to adversarial image inputs.
Multimodal Pragmatic Jailbreak on Text-to-image Models
LMU Munich, Germany(慕尼黑大学);Munich Center for Machine Learning, Germany(慕尼黑机器学习中心);Saarland University, Germany(萨尔兰大学);Max Planck Institute for Informatics, Germany(马克斯·普朗克信息研究所);Cornell University, USA(康奈尔大学);University of Oxford, UK(牛津大学)
PixJail: Self-Evolving Paper-to-Pipeline Reproduction for Text-to-Image Jailbreak Evaluation
The Hong Kong University of Technology and Science (Guangzhou)(香港科技与应用科技大学(广州));East China Normal University(华东师范大学);Shanghai Qi Zhi Institute(上海启智研究院);Wuhan University(武汉大学);Institute of Deep Perception Technology, JITRI(视觉感知技术研究院,JITRI);CAIR, Hong Kong Institute of Science and Innovation (HKISI)(创新科技研究院,香港科学与创新研究院(HKISI));MAIS, Institute of Automation, Chinese Academy of Sciences(自动化研究所,中国科学院)
Heuristic-Induced Multimodal Risk Distribution Jailbreak Attack for Multimodal Large Language Models
Sun Yat-Sen University(中山大学);Zhejiang University(浙江大学);BraneMatrix AI;Nanyang Technological University(南洋理工大学);Alibaba Group(阿里巴巴集团);Renmin University of China(中国人民大学)
Align is not Enough: Multimodal Universal Jailbreak Attack against Multimodal Large Language Models
School of Computer Science and Information Engineering, Hefei University of Technology(合肥工业大学计算机科学与信息工程学院);Tsinghua University(清华大学);Institute of automation, Chinese academy of science(中国科学院自动化研究所);School of Computer Science and Engineering, Nanyang Technological University(南洋理工大学计算机科学与工程学院)
MMJailBench: A Factorized Benchmark for Disentangling Multimodal Jailbreak Vulnerabilities
Tongji University(同济大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)
Red-teaming the Multimodal Reasoning: Jailbreaking Vision-Language Models via Cross-modal Entanglement Attacks
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);People’s Public Security University of China(中国人民公安大学)
Test-Time Immunization: A Universal Defense Framework Against Jailbreaks for (Multimodal) Large Language Models
NLPR & MAIS, Institute of Automation, Chinese Academy of Sciences(国家工程实验室与人工智能研究所,中国科学院);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
PiCo: Jailbreaking Multimodal Large Language Models via Pictorial Code Contextualization
School of Artificial Intelligence, Wuhan University(武汉大学人工智能学院);School of Electronic and Computer Engineering, Peking University(北京大学电子与计算机工程学院);Beijing Academy of Artificial Intelligence(北京人工智能研究院)
SafeMobile: Chain-level Jailbreak Detection and Automated Evaluation for Multimodal Mobile Agents
Nanyang Technological University(南洋理工大学);Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所);Beihang University(北京航空航天大学);Sun Yat-sen University(中山大学);Beijing Institute of Technology(北京理工大学);National University of Singapore(新加坡国立大学)
SafePTR: Token-Level Jailbreak Defense in Multimodal LLMs via Prune-then-Restore Mechanism
Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China(电子科技大学深圳研究院);Southwestern University of Finance and Economics(西南财经大学);Engineering Research Center of Intelligent Finance, Ministry of Education(教育部智能金融工程研究中心);Tongji University(同济大学)
PolyJailbreak: Cross-Modal Jailbreaking Attacks on Black-Box Multimodal LLMs
School of Cyber Science and Engineering, Sichuan University(四川大学网络空间安全学院);School of Cyber Security, Tianjin University(天津大学网络安全学院);College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院)
Universally Unfiltered and Unseen:Input-Agnostic Multimodal Jailbreaks against Text-to-Image Model Safeguards
Information Engineering University Zhengzhou China;School of Computer Science, \ University Wuhan China;Xi’an Jiaotong University Xi’an China;Wuhan University Wuhan China;Information Engineering University;School of Computer Science, \ University;Xi’an Jiaotong University;Wuhan University
MIRAGE: Multimodal Immersive Reasoning and Guided Exploration for Red-Team Jailbreak Attacks
University of Waterloo(滑铁卢大学);National University of Singapore(新加坡国立大学);University of California, Merced(加州大学默塞德分校);University of Alberta(阿尔伯塔大学);University of Queensland(昆士兰大学)
AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Prompting
Peking University(北京大学);University of Wisconsin–Madison(威斯康星大学麦迪逊分校);International Digital Economy Academy(国际数字经济学院);University of California Davis(加州大学戴维斯分校)
Jailbreaking LLMs & VLMs: Mechanisms, Evaluation, and Unified Defense
School of Cyberspace Security, Beijing University of Posts and Telecommunications(北京邮电大学网络空间安全学院);China Academy of Information and Communications Technology(中国信息通信研究院)
MIDAS: Multi-Image Dispersion and Semantic Reconstruction for Jailbreaking MLLMs
Beijing University of Posts and Telecommunications(北京邮电大学);Nanyang Technological University(南洋理工大学);Guilin University of Electronic Technology(桂林电子科技大学)
Secure Tug-of-War (SecTOW): Iterative Defense-Attack Training with Reinforcement Learning for Multimodal Model Security
Institute of Artificial Intelligence (TeleAI), China Telecom, China(人工智能研究院(TeleAI),中国电信,中国);Northwestern Polytechnical University(西北工业大学);China Telecom, China(中国电信,中国)
Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignment
University of Maryland(马里兰大学);Indian Institute of Technology Bombay(印度班加罗尔理工学院);Princeton University(普林斯顿大学);University of Colorado Boulder(科罗拉多大学博尔德分校);Capital One(Capital One公司);University of Central Florida(佛罗里达中央大学)