Automated red-teaming methods for large language models typically optimize attack prompts within a fixed, human-designed strategy, leaving the attack strategy itself unchanged. We instead optimize the strategy. We propose AutoRISE, a method that searches over executable attack programs rather than individual prompts. At each iteration, a coding agent edits a strategy and a fixed evaluation harness scores the resulting attacks, returning both a scalar objective and per-example diagnostics that guide subsequent edits. This allows structural changes, including new attack components and altered control flow, that prompt-level methods do not directly express. We also release two benchmark suites developed on disjoint target sets and evaluate on 11 models from five families against seven established jailbreak datasets. Across held-out models, AutoRISE improves average attack success rate by 17.0 points over the strongest baseline, and improves attack success by up to 16 points on frontier targets with low baseline success rates. Ablations against parametric and strategy-library baselines suggest that these gains arise from unrestricted program search, particularly compositional techniques and control-flow edits. AutoRISE operates in a black-box, inference-only setting, requiring no fine-tuning, human annotation, or GPU compute.
Holistic Automated Red Teaming for Large Language Models through Top-Down Test Case Generation and Multi-turn Interaction
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院)
RedAgent: Red Teaming Large Language Models with Context-aware Autonomous Language Agent
The State Key Laboratory of Blockchain and Data Security, Zhejiang University(浙江大学区块链与数据安全国家重点实验室);School of Cyber Science and Technology, Zhejiang University(浙江大学网络科学与技术学院);Palo Alto Networks(派拓网络);University of North Texas(北得克萨斯大学)
RedTopic: Toward Topic-Diverse Red Teaming of Large Language Models
Fudan University(复旦大学);City University of Hong Kong(香港城市大学);Deakin University(德克萨斯大学);Hon Hai Research Institute(鸿海研究室);Hong Kong University of Science and Technology(香港理工大学)
Red-teaming the Multimodal Reasoning: Jailbreaking Vision-Language Models via Cross-modal Entanglement Attacks
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);People’s Public Security University of China(中国人民公安大学)
University of Toronto(多伦多大学);Vector Institute for AI(人工智能向量研究所);Johns Hopkins Medical Institutions(约翰霍普金斯医学机构);University Health Network(医疗网络);Algoma University(阿尔戈马大学);University of Iowa Hospitals & Clinics(爱荷华大学医院与诊所);University of Edinburgh(爱丁堡大学)
Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models
National Key Laboratory for Novel Software Technology, Nanjing University(新型软件技术国家实验室,南京大学);School of Artificial Intelligence, Nanjing University(人工智能学院,南京大学);The Hong Kong University of Science and Technology(香港科技大学);The Chinese University of Hong Kong, Shenzhen(香港大学深圳校区);Southern University of Science and Technology(南方科技大学)
Arondight: Red Teaming Large Vision Language Models with Auto-generated Multi-modal Jailbreak Prompts
City University of Hong Kong(香港城市大学);City University of Hong Kong (Dongguan)(香港城市大学(东莞));University of Science and Technology Beijing(北京科技大学);The University of Melbourne(墨尔本大学)
Vijil / AI Risk and Vulnerability Alliance(Vijil / AI风险与漏洞联盟);AI Risk and Vulnerability Alliance(AI风险与漏洞联盟);Montreal AI Ethics Institute(蒙特利尔人工智能伦理研究所)
Automatic Red Teaming LLM-based Agents with Model Context Protocol Tools
College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);Palo Alto Networks(帕洛阿尔托网络公司);School of Software Technology, Zhejiang University(浙江大学软件技术学院)
MAStrike: Shapley-Guided Collusive Red-Teaming on Multi-Agent Systems
University of Illinois, Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Virtue AI;University of Chicago(芝加哥大学);Wells Fargo(摩根大通);Salesforce;University of California, Santa Barbara(加州大学圣芭芭拉分校)
DiffZOO: A Purely Query-Based Black-Box Attack for Red-teaming Text-to-Image Generative Model via Zeroth Order Optimization
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Drexel University(德雷塞尔大学);Shanghai Innovation Center for Processor Technologies, SHIC(上海处理器技术创新中心)
Whispers of Wealth: Red-Teaming Google's Agent Payments Protocol via Prompt Injection
School of Computing University of Georgia Athens, USA(佐治亚大学计算机学院 美国亚特兰大);Department of Information Technology Kennesaw State University Kennesaw, USA(凯斯韦尔州立大学信息科技学院 美国凯斯韦尔)
Capability-Based Scaling Trends for LLM-Based Red-Teaming
ELLIS Institute Tübingen(图宾根ELLIS研究所);Max Planck Institute for Intelligent Systems(智能系统马克斯·普朗克研究所);Tübingen AI Center(图宾根人工智能中心);Foundation AI – Cisco Systems Inc.(AI基础研究机构——思科系统公司);EPFL(苏黎世联邦理工学院)