Retrieval-Augmented Generation (RAG) systems enhance large language models (LLMs) by incorporating external knowledge bases, but this may expose them to extraction attacks, leading to potential copyright and privacy risks. However, existing extraction methods typically rely on malicious inputs such as prompt injection or jailbreaking, making them easily detectable via input- or output-level detection. In this paper, we introduce Implicit Knowledge Extraction Attack (IKEA), which conducts Knowledge Extraction on RAG systems through benign queries. Specifically, IKEA first leverages anchor concepts-keywords related to internal knowledge-to generate queries with a natural appearance, and then designs two mechanisms that lead anchor concepts to thoroughly "explore" the RAG's knowledge: (1) Experience Reflection Sampling, which samples anchor concepts based on past query-response histories, ensuring their relevance to the topic; (2) Trust Region Directed Mutation, which iteratively mutates anchor concepts under similarity constraints to further exploit the embedding space. Extensive experiments demonstrate IKEA's effectiveness under various defenses, surpassing baselines by over 80% in extraction efficiency and 90% in attack success rate. Moreover, the substitute RAG system built from IKEA's extractions shows comparable performance to the original RAG and outperforms those based on baselines across multiple evaluation tasks, underscoring the stealthy copyright infringement risk in RAG systems.
Multi-Stage Prompt Inference Attacks on Enterprise LLM Systems
Ukrainian State University of Science and Technologies, ESI "Prydniprovska State Academy of Civil Engineering and Architecture(乌克兰科学与技术国家大学,埃斯I普里皮亚特斯卡州立建筑与工程学院);Google DeepMind(谷歌DeepMind)
AgentSentry: Mitigating Indirect Prompt Injection in LLM Agents via Temporal Causal Diagnostics and Context Purification
Key Laboratory of Aerospace Information Security and Trusted Computing, Ministry of Education, School of Cyber Science and Engineering, Wuhan University(航空信息安全与可信计算重点实验室、教育部、网络安全科学与工程学院、武汉大学);Department of Computer Science and Engineering, University at Buffalo, SUNY(计算机科学与工程系、布法罗大学、纽约州立大学);Department of Computer Science, College of Information Science and Technology, Jinan University(计算机科学系、信息科学与技术学院、济南大学);College of Cyber Security, Jinan University(网络安全学院、济南大学)
Joint Optimization of Prompt Security and System Performance in Edge-Cloud LLM Systems
United International College (UIC)(联合国际学院);Institute of Artificial Intelligence and Future Networks (BNU-Zhuhai)(人工智能与未来网络研究院(北京师范大学珠海校区));Interdisciplinary Intelligence SuperComputer Center of Beijing Normal University (Zhuhai)(北京师范大学(珠海)跨学科智能超级计算中心)
Semantic Intent Fragmentation: A Single-Shot Compositional Attack on Multi-Agent AI Pipelines
Department of Computer Science, University of Texas at El Paso(德克萨斯理工大学计算机科学系);School of Computing, Southern Illinois University Carbondale(南方伊利诺伊大学卡本代尔分校计算机学院);Hanyang University(翰阳大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
Design and Implementation of a Secure RAG-Enhanced AI Chatbot for Smart Tourism Customer Service: Defending Against Prompt Injection Attacks -- A Case Study of Hsinchu, Taiwan
Department of Information Management, National Dong Hwa University(信息管理系,国立东华大学);By The Student (BTS) Experimental Education Program (Non-School Type)(学生(BTS)实验教育计划(非学校类型))