Automated jailbreak attacks rely on evaluation feedback to optimize adversarial prompts, but existing methods often lack fine-grained diagnostic information. We propose Jev-Guided Strategy Bandit (JSB), a query-based jailbreak framework that leverages multi-dimensional probabilistic feedback to guide prompt optimization. JSB combines continuous jailbreak scoring with targeted refinement, strategy switching, and best-state reseeding to balance exploration and refinement. We evaluate JSB on two harmful request datasets across five open-source and three closed-source LLMs. Under a budget of only 10 target-model queries, JSB achieves an average HarmBench attack success rate of 89.8\% across ten open-source settings, consistently outperforming existing automated jailbreak attacks. Further analyses demonstrate its effectiveness under varying query budgets and attacker models. Defense experiments also reveal limitations in existing filtering mechanisms and highlight the potential of Jev-based output filtering.
D-Judge: Disrupting Multi-Turn Jailbreaks using Semantics-Preserving Output Rewriting
University of California, Berkeley(加州大学伯克利分校);International Computer Science Institute(国际计算机科学研究所);University of California, Riverside(加州大学河滨分校);Lawrence Berkeley National Laboratory(伯克利国家实验室)
PandaGuard: Systematic Evaluation of LLM Safety against Jailbreaking Attacks
Beijing Institute of AI Safety and Governance (Beijing-AISI)(北京人工智能安全与治理研究院);Beijing Key Laboratory of Safe AI and Superalignment(北京安全人工智能与超对齐重点实验室);BrainCog Lab, CASIA(CASIA脑认知实验室)
Measuring the Wrong Thing: Internal Harmfulness Scores Anti-Rank Successful Jailbreaks
College of Computer Science and Artificial Intelligence, Fudan University(复旦大学计算机科学与人工智能学院);School of Computer Engineering and Science, Shanghai University(上海大学计算机工程与科学学院);Beijing Normal University(北京师范大学);Institute of Advanced Intelligence and Computing, A*STAR(新加坡科技研究局高级智能与计算研究所);School of Communication and Electronic Engineering, East China Normal University(华东师范大学通信与电子工程学院);Institute of Big Data, Fudan University(复旦大学大数据研究院);Department of Computer Science, Worcester Polytechnic Institute(伍斯特理工学院计算机科学系)
Efficient LLM Safety Evaluation through Multi-Agent Debate
Beijing Institute of AI Safety and Governance(北京人工智能安全与治理研究院);Beijing Key Laboratory of Safe AI and Super Alignment(北京安全人工智能与超对齐重点实验室);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);CSE, The Chinese University of Hong Kong(香港中文大学电子工程系);University of Chinese Academy of Sciences(中国科学院大学);Department of Mathematics, The Chinese University of Hong Kong(香港中文大学数学系);Long-term AI(长期人工智能)
Red-TTT: Test-Time Training for Automated Jailbreaking Large Language Models
Johns Hopkins University(约翰斯·霍普金斯大学);National University of Singapore(新加坡国立大学);Washington University in St. Louis(圣路易斯华盛顿大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Stanford University(斯坦福大学)