arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

提示侧智能体剧本何时可迁移?智能体部署中的准确性、成本与运行时偏移

When Do Prompt-Side Agent Playbooks Transfer? Accuracy, Cost, and Runtime Shift in Agent Deployment

Weihong Lin, Lin Sun, Xiangzheng Zhang

arXiv 2608.05778首次发表:更新:

发表机构

Beijing Qiyuan Technology Co., Ltd.(北京奇源科技有限公司)

机构由 AI 辅助整理,请以论文原文为准。

AI 中文总结

该研究在蒸馏-验证-迁移协议下探究提示侧智能体剧本的可迁移性,在ALFWorld、TAU2-Bench、XBench-DeepSearch等基准上测试发现其为有条件的冷启动选项,需目标侧验证相关指标。

AI 中文摘要

提示侧剧本无需重新训练即可提升使用工具的语言智能体的性能,但其在源设置之外的可迁移性尚不明确。我们在共享的蒸馏-验证-迁移协议下研究冻结剧本的迁移。在ALFWorld上,在受控贪心解码下迁移是有益的,且在一项接近预算匹配的比较中,蒸馏指导优于5个固定演示。在TAU2-Bench上,预先指定的总体对比支持适度的平均匹配域优势,但全局Holm校正在135个路线级效应中仅保留1个;剩余网格提供了兼容性敏感异质性的描述性证据。在XBench-DeepSearch上,一种人工制品-运行时配对在保留有用的首次尝试启发式的同时,在上下文-运行时偏移后产生重复查询、延迟终止和大幅成本膨胀。在所有基准上,迁移的和目标衍生的剧本都需要在目标侧验证成功、终止、协议兼容性和成本。因此,冻结迁移是有条件的冷启动选项,而非默认复用策略或优于目标侧重新蒸馏的普遍优选方案。

英文摘要

Prompt-side playbooks can improve tool-using language agents without retraining, but their portability beyond the source setting is unclear. We study frozen playbook transfer under a shared distill--validate--transfer protocol. On ALFWorld, transfer is beneficial under controlled greedy decoding and, in one near-budget-matched comparison, distilled guidance outperforms five fixed demonstrations. On TAU2-Bench, a prespecified aggregate contrast supports a modest average matched-domain advantage, but global Holm correction retains only one of 135 route-level effects; the remaining grid provides descriptive evidence of compatibility-sensitive heterogeneity. On XBench-DeepSearch, one artifact--runtime pairing preserves useful first-try heuristics while producing repeated queries, delayed stopping, and substantial cost inflation after a context-runtime shift. Across benchmarks, transferred and target-derived playbooks both require target-side validation of success, termination, protocol compatibility, and cost. Frozen transfer is therefore a conditional cold-start option, not a reuse-by-default strategy or a universally preferable alternative to target-side redistillation.

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑