提示注入作为新兴威胁：评估大型语言模型的 resilience
Prompt Injection as an Emerging Threat: Evaluating the Resilience of Large Language Models
AI总结：
本研究针对提示注入这一新兴威胁，提出统一评估框架（含RDI、SCC、IIM指标），测试GPT-4等4个LLM在5类任务中的抗性，发现GPT-4表现最优，对齐与安全调优比模型大小更影响抗性，为提升LLM安全提供实用框架。
摘要：
大型语言模型（LLMs）越来越多地用于执行推理、摘要和代码生成的智能系统中。它们遵循自然语言指令的能力虽然强大，但也使它们容易受到一类称为提示注入的新型攻击。在这些攻击中，隐藏或恶意指令被插入到用户输入或外部内容中，导致模型忽略其预期任务或产生不安全的响应。本研究提出了一个统一框架，用于评估大型语言模型（LLMs）对提示注入攻击的抵抗能力。该框架定义了三个互补指标，如Resilience Degradation Index（RDI，弹性退化指数）、Safety Compliance Coefficient（SCC，安全合规系数）和Instructional Integrity Metric（IIM，指令完整性指标），以联合衡量鲁棒性、安全性和语义稳定性。我们在五个常见语言任务（问答、摘要、翻译、推理和代码生成）上评估了四个指令调优模型（GPT-4、GPT-4o、LLaMA-3 8B Instruct和Flan-T5-Large）。结果显示，GPT-4总体表现最佳，而开源模型仍然更脆弱。研究结果强调，强大的对齐和安全调优对于弹性比模型大小本身更重要。结果表明，所有模型仍然部分脆弱，特别是对间接和直接覆盖攻击。GPT-4实现了最佳的总体弹性（RDR = 9.8％，SCR = 96.4％），而开源模型表现出更高的性能下降和更低的安全分数。研究结果表明，对齐强度和安全调优在弹性方面比模型大小本身发挥更大的作用。所提出的框架提供了一种结构化、可重现的方法来评估模型的鲁棒性，并为提高LLM的安全性和可靠性提供了实用见解。
英文摘要：
Large Language Models (LLMs) are increasingly used in intelligent systems that perform reasoning, summarization, and code generation. Their ability to follow natural-language instructions, while powerful, also makes them vulnerable to a new class of attacks known as prompt injection. In these attacks, hidden or malicious instructions are inserted into user inputs or external content, causing the model to ignore its intended task or produce unsafe responses. This study proposes a unified framework for evaluating how resistant Large Language Models (LLMs) are to prompt injection attacks. The framework defines three complementary metrics such as the Resilience Degradation Index (RDI), Safety Compliance Coefficient (SCC), and Instructional Integrity Metric (IIM) to jointly measure robustness, safety, and semantic stability. We evaluated four instruction-tuned models (GPT-4, GPT-4o, LLaMA-3 8B Instruct, and Flan-T5-Large) on five common language tasks: question answering, summarization, translation, reasoning, and code generation. Results show that GPT-4 performs best overall, while open-weight models remain more vulnerable. The findings highlight that strong alignment and safety tuning are more important for resilience than model size alone. Results show that all models remain partially vulnerable, especially to indirect and direct-override attacks. GPT-4 achieved the best overall resilience (RDR = 9.8 %, SCR = 96.4 %), while open-source models exhibited higher performance degradation and lower safety scores. The findings demonstrate that alignment strength and safety tuning play a greater role in resilience than model size alone. The proposed framework offers a structured, reproducible approach for assessing model robustness and provides practical insights for improving LLM safety and reliability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
