Jailbreak attacks represent one of the most sophisticated threats to the security of large language models (LLMs). To deal with such risks, we introduce an innovative framework that can help evaluate the effectiveness of jailbreak attacks on LLMs. Unlike traditional binary evaluations focusing solely on the robustness of LLMs, our method assesses the attacking prompts' effectiveness. We present two distinct evaluation frameworks: a coarse-grained evaluation and a fine-grained evaluation. Each framework uses a scoring range from 0 to 1, offering unique perspectives and allowing for the assessment of attack effectiveness in different scenarios. Additionally, we develop a comprehensive ground truth dataset specifically tailored for jailbreak prompts. This dataset is a crucial benchmark for our current study and provides a foundational resource for future research. By comparing with traditional evaluation methods, our study shows that the current results align with baseline metrics while offering a more nuanced and fine-grained assessment. It also helps identify potentially harmful attack prompts that might appear harmless in traditional evaluations. Overall, our work establishes a solid foundation for assessing a broader range of attack prompts in prompt injection.
School of Computer Science and Technology, Guangdong University of Technology（广东工业大学计算机科学与技术学院）；School of Computer Science and Intelligence Education, Lingnan Normal University（岭南师范学院计算机与智能教育学院）；College of Computer, National University of Defense Technology（国防科技大学计算机学院）；School of Information Science and Technology, Guangdong University of Foreign Studies（广东外语外贸大学信息科学与技术学院）
A Mousetrap: Fooling Large Reasoning Models for Jailbreak with Chain of Iterative Chaos
Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；The University of Hong Kong（香港大学）；Fudan University（复旦大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
Harnessing Task Overload for Scalable Jailbreak Attacks on Large Language Models
School of Future Technology, University of Chinese Academy of Sciences（中国科学院大学未来技术学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Beijing Institute of AI Safety and Governance（北京人工智能安全与治理研究院）；Center for Long-term Artificial Intelligence（长期人工智能中心）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Siren: A Learning-Based Multi-Turn Attack Framework for Simulating Real-World Human Jailbreak Behaviors
Department of Computing The Hong Kong Polytechnic University Hong Kong SAR, China；Centre for Artificial Intelligence；Robotics Hong Kong Institute of Science \& Innovation Chinese Academy of Sciences Hong Kong SAR, China
