Multimodal large language models (MLLMs) have advanced the capabilities to interpret and act on visual input in 3D environments, empowering diverse applications such as robotics and situated conversational agents. When MLLMs reason over camera-captured views of the physical world, a new attack surface emerges: an attacker can place text-bearing physical objects in the environment to override MLLMs' intended task. While prior work has studied prompt injection in the text domain and through digitally edited 2D images, limited attention has been paid to how these attacks function in 3D environments. To bridge the gap, we introduce PI3D, a prompt injection attack against MLLMs in 3D environments, realized through text-bearing object placement rather than digital image edits. We formulate and solve the problem of identifying an effective pose (position and orientation) for a 3D object with injected text, where the attacker's goal is to induce the MLLM to perform the injected task while ensuring that the object placement remains physically plausible. Experiment results demonstrate that PI3D is an effective attack against multiple MLLMs under diverse camera trajectories. We further evaluate a range of defenses and show that they are not sufficient to reliably defend against PI3D.
Physical Prompt Injection Attacks on Large Vision-Language Models
School of Cyber Science and Engineering, Wuhan University(武汉大学计算机科学与工程学院);College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院);College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics(南京航空航天大学计算机科学与技术学院)
The Promptware Kill Chain: How Prompt Injections Gradually Evolved Into a Multistep Malware Delivery Mechanism
Department of Software and Information Systems Engineering, Ben-Gurion University of the Negev(本·古里安大学软件与信息系统工程系);School of Electrical and Computer Engineering, Tel Aviv University(特拉维夫大学电气与计算机工程学院);Harvard Kennedy School, Harvard University, and Munk School, University of Toronto(哈佛大学哈佛肯尼迪学校及多伦多大学穆克学校)
Soft Begging: Modular and Efficient Shielding of LLMs against Prompt Injection and Jailbreaking based on Prompt Tuning
Deutsches Forschungszentrum für Künstliche Intelligenz (DFKI)(德国人工智能研究中心);Centre for European Research in Trusted AI (CERTAIN)(欧洲可信人工智能研究中心);sequire technology GmbH(sequire技术有限公司)
UC Berkeley(加州大学伯克利分校);UC Santa Barbara(加州大学圣巴巴拉分校);Duke University(杜克大学);National University of Singapore(新加坡国立大学);King Abdulaziz City for Science and Technology(国王阿卜杜勒阿齐兹城市科学技术学院);University of Washington(华盛顿大学)
Prompt Injection attack against LLM-integrated Applications
Griffith University(格里菲斯大学);Nanyang Technological University(南洋理工大学);University of New South Wales(新南威尔士大学);Huazhong University of Science and Technology(华中科技大学);Indiana University at Bloomington(印第安纳大学布卢明顿分校);Southern University of Science and Technology(南方科技大学);Tianjin University(天津大学)
Transferable Direct Prompt Injection via Activation-Guided MCMC Sampling
School of Software Engineering, Huazhong University of Science and Technology(华中科技大学软件学院);School of Cyber Science and Engineering, Huazhong University of Science and Technology(华中科技大学网络安全学院);College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院);Faculty of Data Science, City University of Macau(澳门城市大学数据科学学院)
Measuring Real-World Prompt Injection Attacks in LLM-based Resume Screening
University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);Duke University(杜克大学);Arizona State University(亚利桑那州立大学);hireEZ;University of California, Berkeley(加州大学伯克利分校)
Divide and Inject: Can Agents Reconstruct an Indirect Prompt Injection from Fragments?
International Computer Science Institute(国际计算机科学研究所);DSO National Laboratories(国防科技局国家实验室);UC Riverside(加州大学河滨分校);Lawrence Berkeley National Lab(劳伦斯伯克利国家实验室)
When Prompts Become Payloads: A Framework for Mitigating SQL Injection Attacks in Large Language Model-Driven Applications
Hasso Plattner Institute, University of Potsdam, Germany(波茨坦大学霍斯曼-普拉特研究所, 德国);Chemnitz University of Technology, Chemnitz, Germany(Chemnitz技术大学, Chemnitz, 德国)