工具失败后的捏造：工具增强智能体断言其工具未返回的值
Fabrication After Tool Failure: Tool-Augmented Agents Assert Values Their Tools Did Not Return
- The Shishukunj International School（希什昆军国际学校）
- Haileybury Astana（阿斯塔纳黑利伯瑞学校）
- UWCSEA East Campus（东南亚世界联合书院东校区）
- Apta AI（Apta人工智能公司）
- Spark AI Research（星火人工智能研究院）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本研究通过基准测试揭示工具增强语言模型在工具失败后常捏造结果，并提出要求模型声明检索状态的简单提示方法，将不诚实率从14.10%降至0.87%。
AI中文摘要：
工具增强语言模型的评估标准是它们是否得出正确答案，而非在工具未能提供答案时是否如实报告。我们通过一个包含1024个条目、覆盖16个内部系统领域和8种工具失败类型的基准测试来隔离这种失败后的决策，其中工具调用被强制执行，且返回的载荷保证不可用。在部署风格的系统提示下，14.10%的响应是不诚实的：模型要么断言载荷无法支持的值，要么在引用捏造的策略或能力限制时拒绝执行。该比率几乎完全由失败是否被信号化所决定。当工具返回status:error时，不诚实行为不存在（0.0%）；当它返回status:ok但带有被编辑、损坏、过期、格式错误、空或截断的值时，不诚实行为达到45.3%。这种行为并非我们提示的产物：它在中性提示下出现（10.17%），在我们评估的每个生产智能体框架的出厂提示下也出现，在CrewAI的提示下达到24.67%，而我们审计的九个框架中没有一个指定模型在工具失败时应做什么。比较提示级别的防御措施，我们发现关键变量不是对工具输出的遵从，而是缺乏命名的失败状态。附加一个要求模型在回答前发出retrieval_status: OK或FAILED的句子，将不诚实行为从14.10%降至0.87%，其中688个条目中有1个恶化，92个改善，并且该句子不变地转移到三个外部智能体框架中。发出的标志在99.7%-99.9%的声明中是忠实的，提供了一个仅需正则表达式的运行时检测器。
英文摘要：
Tool-augmented language models are evaluated on whether they reach the right answer, not on whether they report honestly when a tool fails to supply one. We isolate this post-failure decision with a benchmark of 1,024 items spanning 16 internal-system domains and eight tool-failure types, in which a tool call is enforced and the returned payload is guaranteed to be unusable. Under a deployment-style system prompt, 14.10% of responses are dishonest: the model either asserts a value the payload cannot support or declines while citing a fabricated policy or capability limit. The rate is governed almost entirely by whether the failure is signalled. When the tool returns status:error, dishonesty is absent (0.0%); when it returns status:ok with a redacted, corrupted, stale, malformed, empty or truncated value, dishonesty reaches 45.3%. The behaviour is not an artefact of our prompts: it appears under a neutral prompt (10.17%) and under the shipped prompt of every production agent framework we evaluate, reaching 24.67% under CrewAI's, and none of the nine frameworks we audit specifies what the model should do when a tool fails. Comparing prompt-level defences, we find that the operative variable is not deference to tool output but the absence of a named failure state. Appending a single sentence that requires the model to emit retrieval_status: OK or FAILED before answering reduces dishonesty from 14.10% to 0.87%, with one item of 688 worsening against 92 improving, and transfers unchanged into three foreign agent scaffolds. The emitted flag is faithful in 99.7-99.9% of declarations, giving a runtime detector that needs only a regular expression.
