arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.03007cs.CLcs.AIcs.CRcs.LG
收藏

BadAgent:在LLM智能体中插入和激活后门攻击

BadAgent: Inserting and Activating Backdoor Attacks in LLM Agents

Zhengzhou University(郑州大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)

Yifei Wang, Dizhan Xue, Shengjie Zhang, Shengsheng Qian

AI总结:

本文提出BadAgent后门攻击方法,通过微调在LLM智能体中嵌入后门,使攻击者能在测试时通过触发器操纵智能体执行有害操作,且攻击在可信数据微调后仍稳健,揭示了基于不可信LLM或数据构建智能体的风险。

摘要:

随着大型语言模型(LLMs)的繁荣,基于LLM的强大智能体已被开发出来,通过一组用户定义的工具提供定制服务。构建LLM智能体的最先进方法采用已训练的LLM,并在智能体任务的数据上进一步微调它们。然而,我们表明此类方法容易受到我们提出的名为BadAgent的后门攻击,在各种智能体任务中,后门可以通过在后门数据上微调来嵌入。在测试时,攻击者可以通过在智能体输入或环境中显示触发器,操纵已部署的LLM智能体执行有害操作。令我们惊讶的是,我们提出的攻击方法即使在可信数据上微调后也极其稳健。尽管后门攻击在自然语言处理中已被广泛研究,但据我们所知,我们可能是首个研究LLM智能体后门攻击的,由于允许使用外部工具,这更加危险。我们的工作展示了基于不可信LLM或数据构建LLM智能体的明确风险。我们的代码公开在https://github.com/DPamK/BadAgent

英文摘要:

With the prosperity of large language models (LLMs), powerful LLM-based intelligent agents have been developed to provide customized services with a set of user-defined tools. State-of-the-art methods for constructing LLM agents adopt trained LLMs and further fine-tune them on data for the agent task. However, we show that such methods are vulnerable to our proposed backdoor attacks named BadAgent on various agent tasks, where a backdoor can be embedded by fine-tuning on the backdoor data. At test time, the attacker can manipulate the deployed LLM agents to execute harmful operations by showing the trigger in the agent input or environment. To our surprise, our proposed attack methods are extremely robust even after fine-tuning on trustworthy data. Though backdoor attacks have been studied extensively in natural language processing, to the best of our knowledge, we could be the first to study them on LLM agents that are more dangerous due to the permission to use external tools. Our work demonstrates the clear risk of constructing LLM agents based on untrusted LLMs or data. Our code is public at https://github.com/DPamK/BadAgent

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑