VLMs (Vision-Language Models) extend the capabilities of LLMs (Large Language Models) to accept multimodal inputs. Since it has been verified that LLMs can be induced to generate harmful or inaccurate content through specific test cases (termed as Red Teaming), how VLMs perform in similar scenarios, especially with their combination of textual and visual inputs, remains a question. To explore this problem, we present a novel red teaming dataset RTVLM, which encompasses 10 subtasks (e.g., image misleading, multi-modal jail-breaking, face fairness, etc) under 4 primary aspects (faithfulness, privacy, safety, fairness). Our RTVLM is the first red-teaming dataset to benchmark current VLMs in terms of these 4 different aspects. Detailed analysis shows that 10 prominent open-sourced VLMs struggle with the red teaming in different degrees and have up to 31% performance gap with GPT-4V. Additionally, we simply apply red teaming alignment to LLaVA-v1.5 with Supervised Fine-tuning (SFT) using RTVLM, and this bolsters the models' performance with 10% in RTVLM test set, 13% in MM-Hal, and without noticeable decline in MM-Bench, overpassing other LLaVA-based models with regular alignment data. This reveals that current open-sourced VLMs still lack red teaming alignment. Our code and datasets will be open-source.
Resource Consumption Red-Teaming for Large Vision-Language Models
China Mobile Research Institute(中国移动研究院);Beijing University of Posts and Telecommunications(北京邮电大学);Nanyang Technological University(南洋理工大学);University of Science and Technology of China(中国科学技术大学);North China Electric Power University(华北电力大学)
Holistic Automated Red Teaming for Large Language Models through Top-Down Test Case Generation and Multi-turn Interaction
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院)
RedTopic: Toward Topic-Diverse Red Teaming of Large Language Models
Fudan University(复旦大学);City University of Hong Kong(香港城市大学);Deakin University(德克萨斯大学);Hon Hai Research Institute(鸿海研究室);Hong Kong University of Science and Technology(香港理工大学)
University of Toronto(多伦多大学);Vector Institute for AI(人工智能向量研究所);Johns Hopkins Medical Institutions(约翰霍普金斯医学机构);University Health Network(医疗网络);Algoma University(阿尔戈马大学);University of Iowa Hospitals & Clinics(爱荷华大学医院与诊所);University of Edinburgh(爱丁堡大学)
RedAgent: Red Teaming Large Language Models with Context-aware Autonomous Language Agent
The State Key Laboratory of Blockchain and Data Security, Zhejiang University(浙江大学区块链与数据安全国家重点实验室);School of Cyber Science and Technology, Zhejiang University(浙江大学网络科学与技术学院);Palo Alto Networks(派拓网络);University of North Texas(北得克萨斯大学)
Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models
National Key Laboratory for Novel Software Technology, Nanjing University(新型软件技术国家实验室,南京大学);School of Artificial Intelligence, Nanjing University(人工智能学院,南京大学);The Hong Kong University of Science and Technology(香港科技大学);The Chinese University of Hong Kong, Shenzhen(香港大学深圳校区);Southern University of Science and Technology(南方科技大学)
Red-teaming the Multimodal Reasoning: Jailbreaking Vision-Language Models via Cross-modal Entanglement Attacks
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);People’s Public Security University of China(中国人民公安大学)
Arondight: Red Teaming Large Vision Language Models with Auto-generated Multi-modal Jailbreak Prompts
City University of Hong Kong(香港城市大学);City University of Hong Kong (Dongguan)(香港城市大学(东莞));University of Science and Technology Beijing(北京科技大学);The University of Melbourne(墨尔本大学)
HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Center for AI Safety(人工智能安全中心);Carnegie Mellon University(卡内基梅隆大学);UC Berkeley(加州大学伯克利分校);Microsoft(微软)
Vijil / AI Risk and Vulnerability Alliance(Vijil / AI风险与漏洞联盟);AI Risk and Vulnerability Alliance(AI风险与漏洞联盟);Montreal AI Ethics Institute(蒙特利尔人工智能伦理研究所)
Red Teaming LLMs as Socio-Technical Practice: From Exploration and Data Creation to Evaluation
Responsible Tech Research IBM Research Yorktown Heights New York USA(负责技术研究 IBM 研究 Yorktown Heights 新 York 美国);IBM Research(IBM 研究);The Pennsylvania State University(宾夕法尼亚州立大学);University of Notre Dame(诺特达美大学)
Red Teaming Models for Hyperspectral Image Analysis Using Explainable AI
Warsaw University of Technology(华沙理工大学);University of Warsaw(华沙大学);Silesian University of Technology(西里西亚工业大学);KP Labs(KP实验室);European Space Agency(欧洲空间局)
DiffZOO: A Purely Query-Based Black-Box Attack for Red-teaming Text-to-Image Generative Model via Zeroth Order Optimization
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Drexel University(德雷塞尔大学);Shanghai Innovation Center for Processor Technologies, SHIC(上海处理器技术创新中心)