As deep learning advances, Large Language Models (LLMs) and their multimodal counterparts, Multimodal Large Language Models (MLLMs), have shown exceptional performance in many real-world tasks. However, MLLMs face significant security challenges, such as jailbreak attacks, where attackers attempt to bypass the model's safety alignment to elicit harmful responses. The threat of jailbreak attacks on MLLMs arises from both the inherent vulnerabilities of LLMs and the multiple information channels that MLLMs process. While various attacks and defenses have been proposed, there is a notable gap in unified and comprehensive evaluations, as each method is evaluated on different dataset and metrics, making it impossible to compare the effectiveness of each method. To address this gap, we introduce \textit{MMJ-Bench}, a unified pipeline for evaluating jailbreak attacks and defense techniques for MLLMs. Through extensive experiments, we assess the effectiveness of various attack methods against SoTA MLLMs and evaluate the impact of defense mechanisms on both defense effectiveness and model utility for normal tasks. Our comprehensive evaluation contribute to the field by offering a unified and systematic evaluation framework and the first public-available benchmark for MLLM jailbreak research. We also demonstrate several insightful findings that highlights directions for future studies.
MMJailBench: A Factorized Benchmark for Disentangling Multimodal Jailbreak Vulnerabilities
Tongji University（同济大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
TeleAI-Safety: A comprehensive LLM jailbreaking benchmark towards attacks, defenses, and evaluations
Institute of Artificial Intelligence (TeleAI) of China Telecom（中国电信人工智能研究院（TeleAI））；Shanghai Jiao Tong University（上海交通大学）；Sichuan University（四川大学）；University of Chinese Academy of Sciences（中国科学院大学）；Chinese Academy of Sciences（中国科学院）；University of Science and Technology of China（中国科学技术大学）；Peking University（北京大学）；Harbin Institute of Technology（哈尔滨工业大学）
DARWIN: Evolving Jailbreak Adversary and Guardrail for LLM Safety Evaluation and Protection
The State Key Laboratory of Blockchain and Data Security, Zhejiang University（浙江大学区块链与数据安全全国重点实验室）；Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security（杭州高新技术开发区（滨江）区块链与数据安全研究所）；Shanghai AI Laboratory（上海人工智能实验室）；East China Normal University（华东师范大学）
Jailbreaking LLMs & VLMs: Mechanisms, Evaluation, and Unified Defense
School of Cyberspace Security, Beijing University of Posts and Telecommunications（北京邮电大学网络空间安全学院）；China Academy of Information and Communications Technology（中国信息通信研究院）
Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMs
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Guangzhou HKUST Fok Ying Tung Research Institute（广州香港科技大学霍英东研究院）；The Hong Kong University of Science and Technology（香港科技大学）；AI Thrust（AI 方向）
BAIT: Boundary-Guided Disclosure Escalation LLM Jailbreaking via Self-Conditioned Reasoning
The Harbin Institute of Technology（哈尔滨工业大学）；The Hong Kong Polytechnic University（香港理工大学）；Shenzhen University（深圳大学）；Shenzhen Loop Area Institute（深圳河套学院）
Jailbreaking for the Average Jane: Choosing Optimal Jailbreaks via Bandit Algorithms for Automatically Enhanced Queries
Centre for Machine Intelligence and Data Science, IIT Bombay（印度理工学院班加罗尔机器智能与数据科学中心）；Department of Computer Science and Engineering, IIT Bombay（印度理工学院班加罗尔计算机科学与工程系）
LLMs Caught in the Crossfire: Malware Requests and Jailbreak Challenges
Institute of Artificial Intelligence (TeleAI), China Telecom（人工智能研究院（TeleAI），中国电信）；Beihang University（北京航空航天大学）；Beijing Jiaotong University（北京交通大学）；Northwestern Polytechnical University（西北工业大学）
JALMBench: Benchmarking Jailbreak Vulnerabilities in Audio Language Models
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；State Key Laboratory of Internet Architecture, Tsinghua University（清华大学互联网架构国家重点实验室）；University of North Texas（北卡罗来纳州立大学）；University of Science and Technology of China（中国科学技术大学）；Fujian Normal University（福建师范大学）；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）
DoubtProbe: Black-Box Jailbreak Defense via Structural Verification and Semantic Auditing
East China Normal University（东华大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）
Jailbreaking the Matrix: Nullspace Steering for Controlled Model Subversion
Department of Computer & Information Science & Engineering, University of Florida（佛罗里达大学计算机与信息科学与工程系）；School of Computer Science, University of Oklahoma（俄克拉荷马大学计算机科学学院）；Computer Science Laboratory, SRI International（SRI国际计算机科学实验室）
GuardNet: Ensemble Strategies of Shallow Neural Networks for Robust Prompt Injection and Jailbreak Detection
Quickium Technology Ltd.（Quickium技术有限公司）；Federal University of São Carlos (UFSCar)（萨尔瓦多·卡罗斯联邦大学）；Federal Institute of Education, Science and Technology of São Paulo (IFSP)（圣保罗教育、科学和技术联邦研究所）
Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignment
University of Maryland（马里兰大学）；Indian Institute of Technology Bombay（印度班加罗尔理工学院）；Princeton University（普林斯顿大学）；University of Colorado Boulder（科罗拉多大学博尔德分校）；Capital One（Capital One公司）；University of Central Florida（佛罗里达中央大学）
