超越提示：测量与优化LLM工具智能体套件
Beyond Prompts: Measuring and Optimizing LLM Tool-Agent Harnesses
- Airbnb, Inc., USA（爱彼迎公司）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本研究提出优化器无关协议测量并优化LLM工具智能体运行时套件，PRISM通过失败聚类和路由在三个基准上平均提升10.1-14.9个百分点。
AI中文摘要：
LLM工具智能体可以通过修改固定模型周围的运行时套件（包括提示、工具接口、中间件、状态处理和恢复逻辑）而无需重新训练即可得到改进。我们将此设置研究为固定模型多轮工具智能体的资源受限套件选择问题，搜索范围限定在提示和工具边界中间件：编辑是工具边界处的受保护拦截，而非对智能体执行逻辑的任意重写。我们的优化器无关协议报告平均留出提升、最差条件提升、可重复性、记录成本诊断以及RelLift95(B)——即在预算B下所选套件留出增益的保守估计。我们使用仅提示和提示加中间件优化器实例化该协议，包括PRISM，它在Pareto搜索中聚类失败并将修复路由到提示、工具边界中间件或联合编辑面。在BFCL多轮、tau2-Retail和tau2-Telecom上，PRISM获得的平均留出提升分别为14.2、14.9和10.1个百分点，并在所有三个基准上获得正的实证RelLift95，组件消融将优势主要归因于失败面路由和编辑模式约束。跨优化器，结果表明某些搜索过程偶尔能发现大增益但仍选择脆弱更新，因此所选套件的可靠性应与平均留出提升一同报告。
英文摘要：
LLM tool agents can be improved without retraining by modifying the runtime harness around a fixed model: prompts, tool interfaces, middleware, state handling, and recovery logic. We study this setting as resource-bounded harness selection for fixed-model multi-turn tool agents, with the search surface scoped to prompts and tool-boundary middleware: edits are guarded intercepts at the tool boundary, not arbitrary rewriting of agent execution logic. Our optimizer-agnostic protocol reports mean held-out lift, worst-condition lift, repeatability, logged cost diagnostics, and RelLift95(B), a conservative estimate of the held-out gain of the harness selected under budget B. We instantiate the protocol with prompt-only and prompt-plus-middleware optimizers, including PRISM, which clusters failures and routes repairs to prompt, tool-boundary middleware, or joint edit surfaces within a Pareto search. On BFCL multi-round, tau2-Retail, and tau2-Telecom, PRISM obtains mean held-out lifts of 14.2, 14.9, and 10.1 percentage points and positive empirical RelLift95 on all three benchmarks, and a component ablation attributes the margin chiefly to failure-surface routing and the edit-pattern constraint. Across optimizers, the results show that some search procedures can occasionally find large gains but still choose brittle updates, so the reliability of the chosen harness should be reported alongside average held-out lift.
