arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2411.14738cs.CLcs.AIcs.CR
收藏

用于精确控制大语言模型输出的通用且上下文无关触发器

Universal and Context-Independent Triggers for Precise Control of LLM Outputs

Tencent(腾讯);Xuanwu Lab(玄武实验室)

Jiashuo Liang, Guancheng Li, Yang Yu

AI总结:

本文提出一种通用且上下文无关的触发器发现方法,可精确操纵大语言模型输出任意指定内容,并揭示了此类攻击对AI智能体决策安全的严重威胁。

摘要:

大语言模型(LLMs)已被广泛应用于自动内容生成乃至关键决策系统等场景。然而,提示注入的风险使得LLM输出存在被操纵的可能。尽管已有大量攻击方法被记录在案,但实现对输出的完全控制仍然困难,通常需要经验丰富的攻击者进行多次尝试,并且高度依赖提示上下文。近期基于梯度的白盒攻击技术在越狱和系统提示泄露等任务中展现出潜力。我们的研究将基于梯度的攻击方法进行推广,以寻找一种满足以下条件的触发器:(1)通用性:无论目标输出是什么都能生效;(2)上下文无关性:在不同提示上下文中均具有鲁棒性;(3)精确输出:能够以高准确率操纵LLM输入以产生任意指定输出。我们提出了一种新方法来高效发现此类触发器,并评估了所提攻击的有效性。此外,我们讨论了此类攻击对基于LLM的应用所带来的重大威胁,特别指出对手可能借此接管AI智能体所做的决策和行动。

英文摘要:

Large language models (LLMs) have been widely adopted in applications such as automated content generation and even critical decision-making systems. However, the risk of prompt injection allows for potential manipulation of LLM outputs. While numerous attack methods have been documented, achieving full control over these outputs remains challenging, often requiring experienced attackers to make multiple attempts and depending heavily on the prompt context. Recent advancements in gradient-based white-box attack techniques have shown promise in tasks like jailbreaks and system prompt leaks. Our research generalizes gradient-based attacks to find a trigger that is (1) Universal: effective irrespective of the target output; (2) Context-Independent: robust across diverse prompt contexts; and (3) Precise Output: capable of manipulating LLM inputs to yield any specified output with high accuracy. We propose a novel method to efficiently discover such triggers and assess the effectiveness of the proposed attack. Furthermore, we discuss the substantial threats posed by such attacks to LLM-based applications, highlighting the potential for adversaries to taking over the decisions and actions made by AI agents.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑