打破MCP：函数劫持攻击：函数调用和代理模型的新威胁
Breaking MCP with Function Hijacking Attacks: Novel Threats for Function Calling and Agentic Models
AI总结：
本文提出了一种新的函数劫持攻击，通过操纵代理模型的工具选择过程，迫使调用特定攻击者选择的函数，展示了代理模型在函数调用接口上的新漏洞。
摘要：
随着代理AI的发展，函数调用大型语言模型（LLMs）受到了广泛关注，这些模型通过调用外部函数来扩展AI系统的能力。注入和劫持攻击已被广泛研究，以展示LLMs对用户提示操纵的脆弱性。代理模型的扩展能力通过其函数调用接口引入了进一步的漏洞。最近的LLM安全研究显示，函数调用可以被滥用，导致数据篡改和盗窃，造成破坏性行为，如无限循环或导致LLMs生成有害内容。本文提出了一种新的函数劫持攻击（FHA），通过操纵代理模型的工具选择过程，迫使调用特定的攻击者选择的函数。尽管现有攻击关注模型在函数调用任务中的语义偏好，我们显示FHA对上下文语义基本无关，并且在功能集上具有鲁棒性，使其适用于不同领域。我们进一步证明FHA可以被训练生成通用对抗函数，使单个被攻击函数能够劫持多个查询和负载配置的工具选择。我们在5种不同模型上进行了实验，包括指导和推理变体，达到了70%至100%的ASR，超过了已建立的BFCL数据集。我们的发现进一步证明了对代理系统需要强大的防护措施和安全模块。
英文摘要：
The growth of agentic AI has drawn significant attention to function calling Large Language Models (LLMs), which are designed to extend the capabilities of AI-powered system by invoking external functions. Injection and jailbreaking attacks have been extensively explored to showcase the vulnerabilities of LLMs to user prompt manipulation. The expanded capabilities of agentic models introduce further vulnerabilities via their function calling interface. Recent work in LLM security showed that function calling can be abused, leading to data tampering and theft, causing disruptive behavior such as endless loops, or causing LLMs to produce harmful content in the style of jailbreaking attacks. This paper introduces a novel function hijacking attack (FHA) that manipulates the tool selection process of agentic models to force the invocation of an attacker-chosen function. While existing attacks focus on semantic preference of the model for function-calling tasks, we show that FHA is largely agnostic to the context semantics and remains effective across domains and function sets. We demonstrate that FHA generalizes to unseen queries and payload perturbations under a fixed target model, reaching 62.5% to 81.9% ASR on held-out queries across 4 function-calling LLMs (instructed and reasoning models), evaluated on the Berkeley Function Calling Leaderboard (BFCL). We further evaluate the cross-model transferability of FHA, showing that FHA can be transferred to other model sizes and families (11.2-27.6% ASR). Our findings further demonstrate the need for strong guardrails and modules for agentic systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
