The Model Context Protocol (MCP) has emerged as a critical standard empowering Large Language Models (LLMs) to utilize external tools. In this ecosystem, LLMs rely on natural language descriptions provided by MCP servers to select and execute functions. This interaction implicitly assumes that tool descriptions faithfully reflect their underlying implementations, while this assumption is not mandatorily verified in practice. As a result, MCP deployments may suffer from a problem named Description-Code Inconsistency (DCI), where a tool's description of its capabilities and security boundaries is not consistent with what the code actually does.
In this paper, we present a comprehensive study of DCI in real-world MCP servers. We formally define the problem and propose a comprehensive taxonomy spanning functionality inconsistencies and undeclared side effects. Guided by this taxonomy, we develop DCIChecker, an automated framework that combines structure-aware static analysis with the Direct-Reverse-Arbitration prompting method to cross-validate tool descriptions against actual code implementations. We apply this framework to a large-scale dataset comprising 19,200 description-code pairs extracted from 2,214 real-world MCP servers. Our measurement reveals that DCI is widespread, with 9.93% of these pairs exhibiting inconsistencies. We further demonstrate that DCI creates a critical defense blind spot, facilitating varied risks from operational failures to stealthy malicious behaviors. Finally, we propose mitigation strategies to enforce semantic consistency and enhance the reliability of the emerging agentic ecosystem.
MCPEvol-Bench: Benchmarking LLM Agent Performance Across Dynamic Evolutions of MCP Servers
College of Computer Science and Technology, National University of Defense Technology(国防科技大学计算机科学与技术学院);State Key Laboratory of Complex & Critical Software Environment(复杂关键软件环境国家重点实验室);National Key Laboratory of Parallel and Distributed Computing(并行与分布式计算国家重点实验室)
Give Them an Inch and They Will Take a Mile:Understanding and Measuring Caller Identity Confusion in MCP-Based AI Systems
Institute for Clarity in Documentation(清晰文档研究所);Inria Paris-Rocquencourt(巴黎-罗克Quantin 研究院);Rajiv Gandhi University(拉朱·甘地大学);Tsinghua University(清华大学);Palmer Research Laboratories(帕勒尔研究实验室);Shandong University(山东省大学);City University of Hong Kong(香港城市大学)
MCP-SafetyBench: A Benchmark for Safety Evaluation of Large Language Models with Real-World MCP Servers
East China Normal University(东华师范大学);Salesforce AI Research(Salesforce人工智能研究);Singapore Management University(新加坡国立大学);Shanghai AI Laboratory(上海人工智能实验室)
Breaking the Protocol: Security Analysis of the Model Context Protocol Specification and Prompt Injection Vulnerabilities in Tool-Integrated LLM Agents
A Taxonomy of Runtime Faults in Model Context Protocol Servers
Jheronimus Academy of Data Science and Tilburg University(赫伦尼姆数据科学学院和蒂尔堡大学);Jheronimus Academy of Data Science(赫伦尼姆数据科学学院);University of Sannio(萨尼亚大学);University of Luxembourg(卢森堡大学);University of Naples Federico II(那不勒斯费德里科二世大学)
Bypassing Document Ingestion: An MCP Approach to Financial Q&A
University of Groningen(Groningen大学);Goethe University Frankfurt(法兰克福歌德大学);Martin Luther University Halle-Wittenberg(哈雷-维滕贝格马丁路德大学);Halle Institute for Economic Research (IWH)(哈雷经济研究所)