Recent advancements in Large Vision-Language Models (VLMs) have underscored their superiority in various multimodal tasks. However, the adversarial robustness of VLMs has not been fully explored. Existing methods mainly assess robustness through unimodal adversarial attacks that perturb images, while assuming inherent resilience against text-based attacks. Different from existing attacks, in this work we propose a more comprehensive strategy that jointly attacks both text and image modalities to exploit a broader spectrum of vulnerability within VLMs. Specifically, we propose a dual optimization objective aimed at guiding the model to generate affirmative responses with high toxicity. Our attack method begins by optimizing an adversarial image prefix from random noise to generate diverse harmful responses in the absence of text input, thus imbuing the image with toxic semantics. Subsequently, an adversarial text suffix is integrated and co-optimized with the adversarial image prefix to maximize the probability of eliciting affirmative responses to various harmful instructions. The discovered adversarial image prefix and text suffix are collectively denoted as a Universal Master Key (UMK). When integrated into various malicious queries, UMK can circumvent the alignment defenses of VLMs and lead to the generation of objectionable content, known as jailbreaks. The experimental results demonstrate that our universal attack strategy can effectively jailbreak MiniGPT-4 with a 96% success rate, highlighting the vulnerability of VLMs and the urgent need for new alignment strategies.
PBI-Attack: Prior-Guided Bimodal Interactive Black-Box Jailbreak Attack for Toxicity Maximization
Alibaba Group（阿里巴巴集团）；Beijing Electronic Science and Technology Institute（北京电子科技研究所）；Nanjing University（南京大学）；Renmin University of China（中国人民大学）；Northeastern University（东北大学）；BraneMatrix AI；Nanyang Technological University（南洋理工大学）
Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulation
PRISM AI（PRISM人工智能公司）；Harmony Intelligence（和谐智能公司）；Leap Laboratories（leap实验室）；MIT CSAIL（麻省理工学院计算机科学与人工智能实验室）；ETH AI Center, ETH Zurich（苏黎世联邦理工学院人工智能中心）
Toward Universal and Transferable Jailbreak Attacks on Vision-Language Models
School of Computing and Information Systems, The University of Melbourne, Australia（墨尔本大学计算机与信息系）；School of Computing and Information Systems, Singapore Management University, Singapore（新加坡管理大学计算机与信息系）；School of Information Technology, Deakin University, Australia（德肯大学信息科技系）；Institute of Trustworthy Embodied AI, Fudan University, China（复旦大学可信具身人工智能研究所）
Cross-Modal Obfuscation for Jailbreak Attacks on Large Vision-Language Models
University of Science and Technology of China（中国科学技术大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Institute of High Performance Computing, A*STAR, Singapore（新加坡A*STAR高性能计算研究所）
Obscure but Effective: Classical Chinese Jailbreak Prompt Optimization via Bio-Inspired Search
Nanyang Technological University（南洋理工大学）；BraneMatrix AI；Nanjing University of Science and Technology（南京理工大学）；Northeast University（东北大学）；Renmin University of China（中国人民大学）；Alibaba Group（阿里巴巴集团）；Beihang University（北航）；National University of Singapore（新加坡国立大学）；Zhejiang Lab（浙江实验室）
Speechmatics（Speechmatics公司）；MATS（机器学习对齐理论与实验项目（注：MATS全称为Machine Learning Alignment Theory and Experiments，为独立研究机构，此处保留缩写并补充说明））；UCL（伦敦大学学院）；Stanford University（斯坦福大学）；University of Oxford（牛津大学）；Tangentic（Tangentic公司）；Anthropic（Anthropic公司）
Geneshift: Impact of different scenario shift on Jailbreaking LLM
National University of Singapore（新加坡国立大学）；NUS Graduate School（新加坡国立大学研究生院）；Integrative Sciences and Engineering Programme（整合科学与工程项目）；Institute of Data Science (IDS)（数据科学研究院）；School of Computing（计算机学院）
Mitigating Many-shot Jailbreak Attacks with One Single Demonstration
Zhejiang University（浙江大学）；Nanyang Technological University（南洋理工大学）；Sun Yat-sen University（中山大学）；University of Science and Technology of China（中国科学技术大学）；Virginia Tech（弗吉尼亚理工大学）
Activation-Guided Local Editing for Jailbreaking Attacks
Beihang University（北航）；Hangzhou Innovation Institute（杭州创新研究院）；HKUST（香港科技大学）；South China University of Technology（华南理工大学）；Nanyang Technological University（南洋理工大学）；Kunming University of Science and Technology（昆明理工大学）
Token-Level Constraint Boundary Search for Jailbreaking Text-to-Image Models
School of Artificial Intelligence, Xidian University（西安电子科技大学人工智能学院）；School of Computer Science and Technology, Xidian University（西安电子科技大学计算机科学与技术学院）；Trustworthy and General Artificial Intelligence Laboratory, School of Engineering, Westlake University（之江实验室可信与通用人工智能实验室）
Typical evaluations of fingerprint recognition systems consist of end-to-end black-box evaluations, which assess performance in terms of overall identification or authentication accuracy. However, these black-box tests of system performance do not reveal insights into the performance of the individual modules, including image acquisition, feature extraction, and matching. On the other hand, white-box evaluations, the topic of this paper, measure the individual performance of each constituent module in isolation. While a few studies have conducted white-box evaluations of the fingerprint reader…
Michigan State University（密歇根州立大学）；National Institute of Standards and Technology（美国国家标准与技术研究院）
Prevailing evaluations of fingerprint recognition systems have been performed as end-to-end black-box tests of fingerprint identification or authentication accuracy. However, performance of the end-to-end system is subject to errors arising in any of its constituent modules, including: fingerprint scanning, preprocessing, feature extraction, and matching. Conversely, white-box evaluations provide a more granular evaluation by studying the individual sub-components of a system. While a few studies have conducted stand-alone evaluations of the fingerprint reader and feature extraction modules of…
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）
Existing works have identified the limitation of top-$1$ attack success rate (ASR) as a metric to evaluate the attack strength but exclusively investigated it in the white-box setting, while our work extends it to a more practical black-box setting: transferable attack. It is widely reported that stronger I-FGSM transfers worse than simple FGSM, leading to a popular belief that transferability is at odds with the white-box attack strength. Our work challenges this belief with empirical finding that stronger attack actually transfers better for the general top-$k$ ASR indicated by the interest …
HTS-Attack: Heuristic Token Search for Jailbreaking Text-to-Image Models
Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局）；CFAR；IHPC；Mohammed Bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Nanyang Technological University（南洋理工大学）；Alibaba Group（阿里巴巴集团）；University of Oxford（牛津大学）
Massachusetts Institute of Technology（麻省理工学院）；University of California, Berkeley（加州大学伯克利分校）
Neural networks are known to be vulnerable to adversarial examples. In this note, we evaluate the two white-box defenses that appeared at CVPR 2018 and find they are ineffective: when applying existing techniques, we can reduce the accuracy of the defended models to 0%.
Department of Computer Science, Stanford University（斯坦福大学计算机科学系）；Mitsubishi UFJ Trust Investment Technology Institute（三菱UFJ信托投资技术研究所）；Japan Digital Design（日本数字设计公司（Japan Digital Design））
Adversarial attacks often involve random perturbations of the inputs drawn from uniform or Gaussian distributions, e.g., to initialize optimization-based white-box attacks or generate update directions in black-box attacks. These simple perturbations, however, could be sub-optimal as they are agnostic to the model being attacked. To improve the efficiency of these attacks, we propose Output Diversified Sampling (ODS), a novel sampling strategy that attempts to maximize diversity in the target model's outputs among the generated samples. While ODS is a gradient-based strategy, the diversity off…
Fraunhofer Institute for Cognitive Systems IKS（弗劳恩霍夫认知系统研究所IKS）；Ruhr West University of Applied Sciences（鲁尔西应用科学大学）
Confidence calibration is a major concern when applying artificial neural networks in safety-critical applications. Since most research in this area has focused on classification in the past, confidence calibration in the scope of object detection has gained more attention only recently. Based on previous work, we study the miscalibration of object detection models with respect to image location and box scale. Our main contribution is to additionally consider the impact of box selection methods like non-maximum suppression to calibration. We investigate the default intrinsic calibration of obj…
