arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2412.04415cs.AI
收藏

直击核心:通过直接LLM操纵攻击基于RAG的智能体的简单有效方法

Targeting the Core: A Simple and Effective Method to Attack RAG-based Agents via Direct LLM Manipulation

HydroX AI

Xuying Li, Zhuo Li, Yuji Kosuga, Yasuhiro Yoshida, Victor Bian

AI总结:

本文研究AI智能体中LLM核心的对抗攻击漏洞,通过测试“忽略文档”等简单对抗前缀,验证其可绕过上下文防护迫使LLM生成危险输出,揭示现有防御脆弱性并强调需多层安全措施。

摘要:

由大型语言模型(LLMs)驱动的AI智能体通过实现无缝、自然且上下文感知的通信,改变了人机交互。尽管这些进步极具实用性,但也继承并放大了固有安全风险,如偏见、公平性问题、幻觉、隐私泄露和缺乏透明度。本文研究了一个关键漏洞:针对AI智能体内部LLM核心的对抗攻击。具体而言,我们测试了如下假设:一个看似简单的对抗前缀(如\textit{忽略文档})可通过绕过上下文防护措施,迫使LLMs生成危险或非预期输出。实验表明该方法攻击成功率(ASR)很高,揭示了现有LLM防御的脆弱性。这些发现强调亟需针对LLM层面和更广泛智能体架构的鲁棒多层安全措施。

英文摘要:

AI agents, powered by large language models (LLMs), have transformed human-computer interactions by enabling seamless, natural, and context-aware communication. While these advancements offer immense utility, they also inherit and amplify inherent safety risks such as bias, fairness, hallucinations, privacy breaches, and a lack of transparency. This paper investigates a critical vulnerability: adversarial attacks targeting the LLM core within AI agents. Specifically, we test the hypothesis that a deceptively simple adversarial prefix, such as \textit{Ignore the document}, can compel LLMs to produce dangerous or unintended outputs by bypassing their contextual safeguards. Through experimentation, we demonstrate a high attack success rate (ASR), revealing the fragility of existing LLM defenses. These findings emphasize the urgent need for robust, multi-layered security measures tailored to mitigate vulnerabilities at the LLM level and within broader agent-based architectures.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑