技能应记住什么?语言模型代理中成本感知技能重写的质量-成本权衡
What Should a Skill Remember? Quality--Cost Trade-offs in Cost-Aware Skill Rewriting for Language Model Agents
AI总结:
研究语言模型代理中技能重写的质量-成本权衡,提出信息保留策略,在SkillsBench上实现成本降低7%-14.7%且保持验证质量。
摘要:
大型语言模型代理越来越依赖技能:可重用的程序文档,编码工作流程、工具使用、实现模式、验证检查和领域规则。技能重写通常被视为提示压缩,但较短的技能可能通过移除防止探索、调试和恢复的稀疏操作锚点而使代理更昂贵。我们通过这种经济视角研究技能重写。我们的受控框架剖析技能结构,使用信息保留策略重写技能,并在固定任务指令、环境和验证器下评估重写。在SkillsBench上的实验揭示了不同策略间明显的质量-成本权衡:API/代码锚定、工作流保护和规则/公式锚定有利于不同的任务族,没有普遍主导的模板。在主要的留出评估中,学习到的策略将总成本降低7.0%,下游代理令牌成本降低6.0%;在冻结的跨模型迁移中,相应的降低平均为14.7%和13.7%,同时验证器质量保持不变。这些结果将技能设计定位为成本感知的操作知识工程,而非提示压缩。资源:\href{https://github.com/1Reminding/Skill_EE}{SkillEE}。
英文摘要:
Large language model agents increasingly rely on skills: reusable procedural documents encoding workflows, tool use, implementation patterns, validation checks, and domain rules. Skill rewriting is often treated as prompt compression, but shorter skills can make agents more expensive by removing sparse operational anchors that prevent exploration, debugging, and recovery. We study skill rewriting through this economic lens. Our controlled framework profiles skill structure, rewrites skills using information-preservation strategies, and evaluates the rewrites under fixed task instructions, environments, and verifiers. Experiments on SkillsBench reveal distinct quality--cost trade-offs across strategies: API/code anchoring, workflow guarding, and rule/formula anchoring benefit different task families, with no universally dominant template. In the main held-out evaluation, the learned policy reduces total cost by 7.0% and downstream agent-token cost by 6.0%; in frozen cross-model transfer, the corresponding reductions average 14.7% and 13.7%, while verifier quality is preserved. These results position skill design as cost-aware operational knowledge engineering rather than prompt compression. Resources: https://github.com/1Reminding/Skill_EE.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。