Large language models and AI chatbots have been at the forefront of democratizing artificial intelligence. However, the releases of ChatGPT and other similar tools have been followed by growing concerns regarding the difficulty of controlling large language models and their outputs. Currently, we are witnessing a cat-and-mouse game where users attempt to misuse the models with a novel attack called prompt injections. In contrast, the developers attempt to discover the vulnerabilities and block the attacks simultaneously. In this paper, we provide an overview of these emergent threats and present a categorization of prompt injections, which can guide future research on prompt injections and act as a checklist of vulnerabilities in the development of LLM interfaces. Moreover, based on previous literature and our own empirical research, we discuss the implications of prompt injections to LLM end users, developers, and researchers.
Measuring Real-World Prompt Injection Attacks in LLM-based Resume Screening
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Duke University（杜克大学）；Arizona State University（亚利桑那州立大学）；hireEZ；University of California, Berkeley（加州大学伯克利分校）
Soft Begging: Modular and Efficient Shielding of LLMs against Prompt Injection and Jailbreaking based on Prompt Tuning
Deutsches Forschungszentrum für Künstliche Intelligenz (DFKI)（德国人工智能研究中心）；Centre for European Research in Trusted AI (CERTAIN)（欧洲可信人工智能研究中心）；sequire technology GmbH（sequire技术有限公司）
"Give a Positive Review Only": An Early Investigation Into In-Paper Prompt Injection Attacks and Defenses for AI Reviewers
Institute of Information Engineering, CAS（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络安全学院）；The Conversational AI (CoAI) group, DCST, Tsinghua University（清华大学计算机科学与技术系对话式人工智能小组）
Prompt Injection attack against LLM-integrated Applications
Griffith University（格里菲斯大学）；Nanyang Technological University（南洋理工大学）；University of New South Wales（新南威尔士大学）；Huazhong University of Science and Technology（华中科技大学）；Indiana University at Bloomington（印第安纳大学布卢明顿分校）；Southern University of Science and Technology（南方科技大学）；Tianjin University（天津大学）
Design Patterns for Securing LLM Agents against Prompt Injections
Invariant Labs；IBM；EPFL（苏黎世联邦理工学院）；ETH Zurich（苏黎世联邦理工学院）；Swisscom（瑞士通信）；Google（谷歌）；ETH AI Center（苏黎世联邦理工学院人工智能中心）；AppliedAI Institute for Europe（欧洲应用AI研究院）；Microsoft（微软）；Lakera
When Prompts Control Robots: Prompt Injection Attacks in Multi-Agent Robotic Systems
School of Informatics, Computing, and Cyber Systems, Northern Arizona University（北亚利桑那大学信息学、计算与网络安全学院）；Department of Software Science, Tallinn University of Technology（塔林理工大学软件科学系）
The Promptware Kill Chain: How Prompt Injections Gradually Evolved Into a Multistep Malware Delivery Mechanism
Department of Software and Information Systems Engineering, Ben-Gurion University of the Negev（本·古里安大学软件与信息系统工程系）；School of Electrical and Computer Engineering, Tel Aviv University（特拉维夫大学电气与计算机工程学院）；Harvard Kennedy School, Harvard University, and Munk School, University of Toronto（哈佛大学哈佛肯尼迪学校及多伦多大学穆克学校）
A Multi-Agent LLM Defense Pipeline Against Prompt Injection Attacks
School of Computing, Wichita State University, Kansas, USA（威斯康星州立大学计算机学院）；College of Engineering and Computer Sciences, Marshall University, Huntington, WV, USA（马歇尔大学工程与计算机科学学院）；Department of Computer Science and Engineering, University of Rajshahi, Bangladesh（拉贾沙希大学计算机科学与工程系）；Department of Computer Science and Engineering, American International University-Bangladesh, Dhaka, Bangladesh（美国国际大学-孟加拉国计算机科学与工程系）；School of Computer Science and Engineering, The University of Aizu, Aizuwakamatsu, Japan（立命馆大学计算机科学与工程学院）
