本文从系统安全视角详细分析了大型语言模型驱动的 AI 智能体存在的潜在安全漏洞及其成因与影响,并针对每种漏洞提出相应的防御机制,通过实验评估其可行性,以提升 AI 智能体的安全性与可靠性。
摘要:
AI 智能体受大型语言模型推动。AI 智能体可作为智能助手,凭借访问工具及在环境中执行命令的能力,代表用户完成任务。通过研究典型 AI 智能体的工作流,我们提出了若干关于其安全性的担忧。这些潜在漏洞既未被构建智能体的框架所解决,也未被旨在改进智能体的研究所处理。本文从系统安全视角详细识别并描述了这些漏洞,强调其成因与严重影响。此外,我们针对每种漏洞引入了相应的防御机制,并通过设计与实验评估其可行性。总体而言,本文将当前 AI 智能体发展中的安全问题置于语境中,并阐明使 AI 智能体更安全、更可靠的方法。
英文摘要:
AI agents have been boosted by large language models. AI agents can function as intelligent assistants and complete tasks on behalf of their users with access to tools and the ability to execute commands in their environments. Through studying and experiencing the workflow of typical AI agents, we have raised several concerns regarding their security. These potential vulnerabilities are not addressed by the frameworks used to build the agents, nor by research aimed at improving the agents. In this paper, we identify and describe these vulnerabilities in detail from a system security perspective, emphasizing their causes and severe effects. Furthermore, we introduce defense mechanisms corresponding to each vulnerability with design and experiments to evaluate their viability. Altogether, this paper contextualizes the security issues in the current development of AI agents and delineates methods to make AI agents safer and more reliable.
A Security Analysis of Long-Horizon Agentic AI Systems: Threats, Evaluation, and Framework Development
Department of Computer Science, College of Computers and Information Technology, Taif University, KSA (Summer 2026)(计算机科学系,计算机与信息科技学院,泰夫大学,沙特阿拉伯(2026年夏季))
Department of Computer Science and Engineering, Southern University of Science and Technology(南方科技大学计算机科学与工程系);Department of Computer Science and Engineering, Hong Kong University of Science and Technology(香港科技大学计算机科学与工程系);Southern University of Science and Technology(南方科技大学);Hong Kong University of Science and Technology(香港科技大学)
Google(谷歌);University of California San Diego(加州大学圣地亚哥分校);University of Wisconsin–Madison(威斯康星大学麦迪逊分校);EmbraceTheRed;Gray Swan AI;Cornell University(康奈尔大学)
SeClaw: Spec-Driven Security Task Synthesis for Evaluating Autonomous Agents
The Hong Kong University of Science and Technology(香港科技大学);Ant Digital Technologies, Ant Group(蚂蚁集团数字技术部);Xi’an Jiaotong University(西安交通大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));University of Oxford(牛津大学);City University of Hong Kong(香港城市大学);Institute of Science Tokyo(东京科学研究所);Zhejiang University(浙江大学);Massachusetts Institute of Technology(麻省理工学院);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);Beijing University of Technology(北京理工大学)
What If Prompt Injection Never Left? Rethinking Agent Security through Cross-Session Stored Prompt Injection
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院);AI Sec Lab, Beijing Chaitin Technology Co.,Ltd(北京柴坦科技有限公司AI安全实验室)
Security Attack and Defense Strategies for Autonomous Agent Frameworks: A Layered Review with OpenClaw as a Case Study
School of Artificial Intelligence and Computer Science, Nantong University(人工智能与计算机科学学院,南通大学);State Key Lab. for Novel Software Technology, Nanjing University(新型软件技术国家重点实验室,南京大学)
From Thinker to Society: Security in Hierarchical Autonomy Evolution of AI Agents
Nanjing University of Aeronautics and Astronautics(南京航空航天大学);Collaborative Innovation Center of Novel Software Technology and Industrialization(新型软件技术与产业化协同创新中心);The Chinese University of Hong Kong(香港中文大学);The University of Hong Kong(香港大学);Zhejiang University(浙江大学);Independent Researcher(独立研究者);Zhejiang Normal University(浙江师范大学);School of Software Technology, Zhejiang University(浙江大学软件学院);Ningbo Global Innovation Center, Zhejiang University(宁波全球创新中心,浙江大学)
Agentic AI for Cyber Resilience: A New Security Paradigm and Its System-Theoretic Foundations
Department of Systems Engineering, City University of Hong Kong(香港城市大学系统工程系);Department of Electrical and Computer Engineering, New York University(纽约大学电气与计算机工程系)
Large Model Based Agents: State-of-the-Art, Cooperation Paradigms, Security and Privacy, and Future Trends
School of Cyber Science and Engineering, Xi’an Jiaotong University(网络科学与工程学院,西安交通大学);School of Automation, Guangdong University of Technology(自动化学院,广东工业大学);College of Computing and Data Science, Nanyang Technological University(计算与数据科学学院,南洋理工大学)
LLM agents security duality: a comprehensive survey of self-security and empowered cybersecurity
School of Cyber Science and Engineering, Wuhan University(武汉大学计算机科学与工程学院);Department of Computer Science and Engineering, University at Buffalo(布法罗大学计算机科学与工程系)
Measuring Security Without Fooling Ourselves: Why Benchmarking Agents Is Hard
ELLIS Institute Tübingen & MPI-IS & Tübingen AI Center(图宾根ELLIS研究所及MPI-IS与图宾根人工智能中心);The Alan Turing Institute(艾伦·图灵研究所);BIFOLD & Technische Universität Berlin(BIFOLD与柏林技术大学);Technische Universität Darmstadt(达姆施塔特技术大学)