发表机构
Georgia Institute of Technology; University of Colorado Boulder(佐治亚理工学院; 科罗拉多大学博尔德分校)
机构由 AI 辅助整理,请以论文原文为准。AI 中文总结
研究揭示 MCP 服务器错误信息面向开发者编写,却使最强智能体恢复率大幅下降,通过在步骤中指明工具或删除步骤可显著提升恢复率。
AI 中文摘要
许多模型上下文协议(MCP)服务器封装了为人类开发者构建的 Web API,其错误信息指示读者运行命令、编辑配置、打开网页或等待。许多读取这些信息的智能体只能调用服务器的工具。在 150 个广泛使用的 MCP 服务器中,3,001 条错误信息里有 949 条告诉调用者下一步该做什么,其中一半的步骤依赖于服务器无法观察到的调用者信息。在凭据错误上,67 个步骤中有 62 个要求执行终端命令、修改配置或打开网页;在速率限制上,30 个步骤中有 20 个说等待并重试,但没有指明要重复的调用。我们测试了五个仅通过伯克利函数调用排行榜任务中的工具行动的 OpenAI 模型,智能体确实执行了步骤所说的操作。在凭据过期时,步骤中的终端命令使 45% 的任务得以恢复,且其造成的损失从 GPT-5.5 的 18 分增加到 GPT-6 Astra 的 69 分。在速率限制时,GitHub 的“等待后重试”仅留下 6% 的恢复率。我们测试了两种补救措施。对于 MCP 开发者,在步骤中指明服务器工具,将凭据过期的恢复率提升至 84%(用登录工具替代命令),速率限制的恢复率提升至 88%(用要重复的调用替代单纯的等待)。对于智能体开发者,在模型读取步骤前用一句话提示删除该步骤,将凭据过期的恢复率提升至 82%。
英文摘要
Many Model Context Protocol (MCP) servers wrap web APIs built for human developers, and their error messages tell the reader to run a command, edit a configuration, open a web page or wait. Many agents that read them can only call the server's tools. In 150 widely used MCP servers, 949 of 3,001 error messages tell the caller what to do next, and half of these steps depend on something the server cannot see about the caller. On credential errors, 62 of 67 steps ask for a terminal command, a configuration change or a web page; on rate limits, 20 of 30 say to wait and retry without naming the call to repeat. We tested five OpenAI models that act only through the tools of Berkeley Function Calling Leaderboard tasks, and the agents did what the step said. On expired credentials, a terminal command in the step left 45% of tasks recovered, and the loss it caused grew from 18 points for GPT-5.5 to 69 for GPT-6 Astra. On a rate limit, GitHub's "Wait before retrying." left 6%. We tested two remedies. For MCP developers, naming a server tool in the step raised recovery on expired credentials to 84%, with the login tool in place of the command, and on a rate limit to 88%, with the call to repeat in place of the bare wait. For agent developers, deleting the step with a one-sentence prompt before the model reads it raised recovery on expired credentials to 82%.
Comments15 pages, 6 tables. Submitted to the Journal of Systems and Software. Data and code: https://github.com/WenJing95/tool-error-text