Arondight: Red Teaming Large Vision Language Models with Auto-generated Multi-modal Jailbreak Prompts
City University of Hong Kong(香港城市大学);City University of Hong Kong (Dongguan)(香港城市大学(东莞));University of Science and Technology Beijing(北京科技大学);The University of Melbourne(墨尔本大学)
Yi Liu, Chengjun Cai, Xiaoli Zhang, Xingliang Yuan, Cong Wang
Large Vision Language Models (VLMs) extend and enhance the perceptual abilities of Large Language Models (LLMs). Despite offering new possibilities for LLM applications, these advancements raise significant security and ethical concerns, particularly regarding the generation of harmful content. While LLMs have undergone extensive security evaluations with the aid of red teaming frameworks, VLMs currently lack a well-developed one. To fill this gap, we introduce Arondight, a standardized red team framework tailored specifically for VLMs. Arondight is dedicated to resolving issues related to the absence of visual modality and inadequate diversity encountered when transitioning existing red teaming methodologies from LLMs to VLMs. Our framework features an automated multi-modal jailbreak attack, wherein visual jailbreak prompts are produced by a red team VLM, and textual prompts are generated by a red team LLM guided by a reinforcement learning agent. To enhance the comprehensiveness of VLM security evaluation, we integrate entropy bonuses and novelty reward metrics. These elements incentivize the RL agent to guide the red team LLM in creating a wider array of diverse and previously unseen test cases. Our evaluation of ten cutting-edge VLMs exposes significant security vulnerabilities, particularly in generating toxic images and aligning multi-modal prompts. In particular, our Arondight achieves an average attack success rate of 84.5\% on GPT-4 in all fourteen prohibited scenarios defined by OpenAI in terms of generating toxic text. For a clearer comparison, we also categorize existing VLMs based on their safety levels and provide corresponding reinforcement recommendations. Our multimodal prompt dataset and red team code will be released after ethics committee approval. CONTENT WARNING: THIS PAPER CONTAINS HARMFUL MODEL RESPONSES.
RedAgent: Red Teaming Large Language Models with Context-aware Autonomous Language Agent
The State Key Laboratory of Blockchain and Data Security, Zhejiang University(浙江大学区块链与数据安全国家重点实验室);School of Cyber Science and Technology, Zhejiang University(浙江大学网络科学与技术学院);Palo Alto Networks(派拓网络);University of North Texas(北得克萨斯大学)
Red-teaming the Multimodal Reasoning: Jailbreaking Vision-Language Models via Cross-modal Entanglement Attacks
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);People’s Public Security University of China(中国人民公安大学)
AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMs
University of Wisconsin–Madison(威斯康星大学麦迪逊分校);NVIDIA(NVIDIA公司);Cornell University(康奈尔大学);Washington University, St. Louis(圣路易斯华盛顿大学);University of Michigan, Ann Arbor(安娜堡大学);The Ohio State University(俄亥俄州立大学);UIUC(伊利诺伊大学厄巴纳-香槟分校)
The Art of the Jailbreak: Formulating Jailbreak Attacks for LLM Security Beyond Binary Scoring
Department of Computer Science, University of Texas at El Paso(德克萨斯大学埃尔帕索分校计算机科学系);School of Computing, Southern Illinois University Carbondale(南方伊利诺伊大学卡罗尔丹分校计算学院);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
SeqAR: Jailbreak LLMs with Sequential Auto-Generated Characters
Shanghai University of Finance and Economics(上海财经大学);Southern University of Science and Technology(南方科技大学);Ant Group(蚂蚁集团);The Chinese University of Hong Kong(香港中文大学)
X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agents
University of California, Los Angeles(加州大学洛杉矶分校);University of Washington(华盛顿大学);Qatar Computing Research Institute(卡塔尔计算研究所);Google(谷歌);Stanford University(斯坦福大学)
MaskForge: Structure-Aware Adaptive Attacks for Jailbreaking Diffusion Large Language Models
University of Wisconsin-Madison(威斯康星大学麦迪逊分校);Johns Hopkins University(约翰霍普金斯大学);University of Southern California(南加州大学);Responsible AI Research (RAIR) Centre, The University of Adelaide(阿德莱德大学负责任人工智能研究中心)
Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMs
Georgia Institute of Technology(佐治亚理工学院);University of Bristol(布里斯托大学);University of Maryland(马里兰大学);University College London(伦敦大学学院);MATS;Astra;MIT CSAIL(麻省理工学院计算机科学与人工智能实验室)
Towards medical AI misalignment: a preliminary study
dept. of Surgical and Medical Science Magna Graecia University of Catanzaro(外科与医学科学系马格拉西亚卡塔尼扎罗大学);dept. of Computer Science Brunel University of London(计算机科学系布鲁内尔伦敦大学);dept. of Computer Engineering, Modelling, Electronics and System University of Calabria(计算机工程、建模、电子与系统系卡塔尼亚大学)
Renmin University of China(中国人民大学);Xi’an Jiaotong University(西安交通大学);University of Science and Technology of China(中国科学技术大学);Wuhan University(武汉大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)
Capability-Based Scaling Trends for LLM-Based Red-Teaming
ELLIS Institute Tübingen(图宾根ELLIS研究所);Max Planck Institute for Intelligent Systems(智能系统马克斯·普朗克研究所);Tübingen AI Center(图宾根人工智能中心);Foundation AI – Cisco Systems Inc.(AI基础研究机构——思科系统公司);EPFL(苏黎世联邦理工学院)
Exploring the Vulnerability of the Content Moderation Guardrail in Large Language Models via Intent Manipulation
Boise State University(博伊州立大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Pittsburgh(匹兹堡大学);New York University(纽约大学);Florida International University(佛罗里达国际大学)
CrossGuard: Safeguarding MLLMs against Joint-Modal Implicit Malicious Attacks
Department of Computer Science, City University of Hong Kong(香港城市大学计算机科学系);Department of Computer Science & Engineering, Washington University in St. Louis(华盛顿大学圣路易斯分校计算机科学与工程系)