The increasing integration of large language models (LLMs) across various fields has heightened concerns about their potential to propagate dangerous information. This paper specifically explores the security vulnerabilities of LLMs within the field of chemistry, particularly their capacity to provide instructions for synthesizing hazardous substances. We evaluate the effectiveness of several prompt injection attack methods, including red-teaming, explicit prompting, and implicit prompting. Additionally, we introduce a novel attack technique named SMILES-prompting, which uses the Simplified Molecular-Input Line-Entry System (SMILES) to reference chemical substances. Our findings reveal that SMILES-prompting can effectively bypass current safety mechanisms. These findings highlight the urgent need for enhanced domain-specific safeguards in LLMs to prevent misuse and improve their potential for positive social impact.
Dagger Behind Smile: Fool LLMs with a Happy Ending Story
S-Lab, Nanyang Technological University, Singapore(南洋理工大学新加坡分校S实验室);College of Computing and Data Science, Nanyang Technological University, Singapore(南洋理工大学计算机与数据科学学院)
Chain-of-Jailbreak Attack for Image Generation Models via Editing Step by Step
Renmin University of China(中国人民大学);Chinese University of Hong Kong(香港大学);Chinese University of Hong Kong, Shenzhen(香港大学(深圳));Johns Hopkins University(约翰霍普金斯大学);Tencent(腾讯);Hong Kong University of Science and Technology(香港科技大学)
In Vino Veritas and Vulnerabilities: Examining LLM Safety via Drunk Language Inducement
School of Computer Science and Engineering, UNSW Sydney(计算机科学与工程学院,新南威尔士大学悉尼分校);School of Computing and Information System, the University of Melbourne(计算与信息系统学院,墨尔本大学)
Evaluation empirique de la sécurisation et de l'alignement de ChatGPT et Gemini: analyse comparative des vulnérabilités par expérimentations de jailbreaks
Geneshift: Impact of different scenario shift on Jailbreaking LLM
National University of Singapore(新加坡国立大学);NUS Graduate School(新加坡国立大学研究生院);Integrative Sciences and Engineering Programme(整合科学与工程项目);Institute of Data Science (IDS)(数据科学研究院);School of Computing(计算机学院)
Jailbreaking for the Average Jane: Choosing Optimal Jailbreaks via Bandit Algorithms for Automatically Enhanced Queries
Centre for Machine Intelligence and Data Science, IIT Bombay(印度理工学院班加罗尔机器智能与数据科学中心);Department of Computer Science and Engineering, IIT Bombay(印度理工学院班加罗尔计算机科学与工程系)
Behind Harmful Compliance: Behavioral and Mechanistic Divergence Across LLM Jailbreaks
Department of Computer Science(计算机科学系);Luddy School of Informatics, Computing, and Engineering(信息学、计算与工程学院);Indiana University Bloomington(印第安纳大学布卢明顿分校)
Leveraging the Potential of Prompt Engineering for Hate Speech Detection in Low-Resource Languages
Bangladesh Institute of Governance and Management (BIGM)(孟加拉国治理与管理研究所);Department of Cyber Security Engineering, George Mason University(网络安全工程系,乔治·梅森大学)
Copyright Detective: A Forensic System to Evidence LLMs Flickering Copyright Leakage Risks
Pine AI;The University of Texas at Austin(德克萨斯大学奥斯汀分校);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Duke University(杜克大学);University of Michigan(密歇根大学);Stevens Institute of Technology(史蒂文斯理工学院);Johns Hopkins University(约翰霍普金斯大学);University of Notre Dame(圣母大学);The Pennsylvania State University(宾夕法尼亚州立大学)
Safety Alignment Should Be Made More Than Just A Few Attention Heads
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络安全学院)
HarmAug: Effective Data Augmentation for Knowledge Distillation of Safety Guard Models
KAIST(韩国科学技术院);Theori;Université de Montréal(蒙特利尔大学);Mila – Québec AI Institute(米拉-魁北克人工智能研究所);Technische Hochschule Nürnberg Georg Simon Ohm(纽伦堡应用技术大学);CIFAR AI Chair(加拿大高级研究所人工智能主席项目)