Machine Learning for Detection and Analysis of Novel LLM Jailbreaks
Centre for Artificial Intelligence and Innovation(人工智能与创新中心);Pingla Institute(平拉研究所);Transitional Artificial Intelligence Research Group(过渡人工智能研究组);UNSW(新南威尔士大学)
John Hawkins, Aditya Pramar, Rodney Beard, Rohitash Chandra
Large Language Models (LLMs) suffer from a range of vulnerabilities that allow malicious users to solicit undesirable responses through manipulation of the input text. These so-called jailbreak prompts are designed to trick the LLM into circumventing the safety guardrails put in place to keep responses acceptable to the developer's policies. In this study, we analyse the ability of different machine learning models to distinguish jailbreak prompts from genuine uses, including looking at our ability to identify jailbreaks that use previously unseen strategies. Our results indicate that using current datasets the best performance is achieved by fine tuning a Bidirectional Encoder Representations from Transformers (BERT) model end-to-end for identifying jailbreaks. We visualise the keywords that distinguish jailbreak from genuine prompts and conclude that explicit reflexivity in prompt structure could be a signal of jailbreak intention.
Mass-Scale Analysis of In-the-Wild Conversations Reveals Complexity Bounds on LLM Jailbreaking
Valencian Research Institute for Artificial Intelligence (VRAIN), Universitat Politècnica de València, Valencia, Spain.(瓦伦西亚人工智能研究 institute,瓦伦西亚理工大学,西班牙瓦伦西亚);Department of Computer Science, The University of Chicago, Chicago, USA(计算机科学系,芝加哥大学,美国芝加哥);Center for Automation and Robotics, Spanish National Research Council, Madrid, Spain(自动化与机器人中心,西班牙国家研究委员会,西班牙马德里)
School of Information Management, Wuhan University(武汉大学信息管理学院);Nanyang Technological University(南洋理工大学);Torr Vision Group, University of Oxford(牛津大学Torr视觉组)
City University of Hong Kong(香港城市大学);Rutgers University(罗格斯大学);University of Science and Technology of China(中国科学技术大学);Universiteit van Amsterdam(阿姆斯特丹大学);Baidu Inc(百度公司)
SeqAR: Jailbreak LLMs with Sequential Auto-Generated Characters
Shanghai University of Finance and Economics(上海财经大学);Southern University of Science and Technology(南方科技大学);Ant Group(蚂蚁集团);The Chinese University of Hong Kong(香港中文大学)
Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMs
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Guangzhou HKUST Fok Ying Tung Research Institute(广州香港科技大学霍英东研究院);The Hong Kong University of Science and Technology(香港科技大学);AI Thrust(AI 方向)
Soft Begging: Modular and Efficient Shielding of LLMs against Prompt Injection and Jailbreaking based on Prompt Tuning
Deutsches Forschungszentrum für Künstliche Intelligenz (DFKI)(德国人工智能研究中心);Centre for European Research in Trusted AI (CERTAIN)(欧洲可信人工智能研究中心);sequire technology GmbH(sequire技术有限公司)
Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study
Nanyang Technological University(南洋理工大学);University of New South Wales(新南威尔士大学);Virginia Tech(弗吉尼亚理工大学);Huazhong University of Science and Technology(华中科技大学)
Geneshift: Impact of different scenario shift on Jailbreaking LLM
National University of Singapore(新加坡国立大学);NUS Graduate School(新加坡国立大学研究生院);Integrative Sciences and Engineering Programme(整合科学与工程项目);Institute of Data Science (IDS)(数据科学研究院);School of Computing(计算机学院)
LLMs Caught in the Crossfire: Malware Requests and Jailbreak Challenges
Institute of Artificial Intelligence (TeleAI), China Telecom(人工智能研究院(TeleAI),中国电信);Beihang University(北京航空航天大学);Beijing Jiaotong University(北京交通大学);Northwestern Polytechnical University(西北工业大学)
Confusion is the Final Barrier: Rethinking Jailbreak Evaluation and Investigating the Real Misuse Threat of LLMs
State Key Lab of Processors, Institute of Computing Technology, CAS(中国科学院计算技术研究所状态关键实验室);University of Chinese Academy of Sciences(中国科学院大学);People’s Public Security University of China(中国人民公安大学);Chinese University of Hong Kong(香港大学)
AdaPPA: Adaptive Position Pre-Fill Jailbreak Attack Approach Targeting LLMs
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院)
Adversarial Tuning: Defending Against Jailbreak Attacks for LLMs
AI Thrust(AI 部);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Guangzhou HKUST Fok Ying Tung Research Institute(广州香港科技大学霍英东研究院);The Hong Kong University of Science and Technology(香港科技大学)
Jailbreaking Attacks vs. Content Safety Filters: How Far Are We in the LLM Safety Arms Race?
CISPA Helmholtz Center for Information Security(CISPA 欧洲信息安全中心);Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);Southern University of Science and Technology(南方科技大学)