AI 中文总结
研究人员推出涵盖4类场景的长上下文基准LongPIBench，发现现有提示注入防御在长上下文下漏洞显著，简单攻击即可绕过，该基准可用于系统评估此类防御。
AI 中文摘要
提示注入攻击对现实应用中的大语言模型构成严重安全风险。然而，现有的提示注入基准主要聚焦于短上下文输入，使得长上下文场景下的攻击与防御机制大多未被探索，这一缺口导致当前防御措施的有效性被大幅高估。本文通过推出LongPIBench（一个用于提示注入的长上下文基准）填补该缺口，该基准涵盖论文同行评审、简历筛选、代码评审、邮件摘要4个现实应用场景，为每个场景构建了合成数据集与现实数据集，上下文长度范围从数千到数万个token。在LongPIBench上的评估结果显示，提示注入防御在长上下文场景下存在显著漏洞：即便简单的启发式提示注入攻击也能达到较高成功率，且常能绕过最先进的防御措施。我们希望LongPIBench能作为实用基准，用于在现实长上下文场景下系统评估提示注入防御。
英文摘要
Prompt injection attacks pose a serious security risk to large language models in real-world applications. However, existing prompt injection benchmarks primarily focus on short-context inputs, leaving the attacks and defenses in long-context settings largely unexplored. This gap leads to a substantial overestimation of the effectiveness of current defenses. In this paper, we bridge the gap by introducing LongPIBench, a long-context benchmark for prompt injection covering 4 realistic application scenarios: paper peer review, resume screening, code review, and email summary. For each scenario, we construct a synthetic dataset and a real-world dataset, with context lengths ranging from thousands to tens of thousands of tokens. The evaluation results on LongPIBench reveal significant vulnerabilities of prompt injection defenses under long-context settings: even simple heuristic prompt injection attacks achieve high success rates and frequently bypass state-of-the-art defenses. We hope LongPIBench can serve as a practical benchmark for systematically evaluating prompt injection defenses in realistic long-context scenarios.
CommentsTo appear in Findings of EMNLP'26
