SkillCraft: 能否让大语言模型代理学会灵活使用工具？
SkillCraft: Can LLM Agents Learn to Use Tools Skillfully?
AI总结：
SkillCraft基准通过测试代理形成和重用高阶工具组合的能力，评估大语言模型在工具使用中的效率提升与技能积累。
摘要：
现实中的工具使用代理在长期工作流中运行，具有重复结构和多样的需求，有效行为不仅需要调用基本工具，还需要抽象和重用更高层次的工具组合。然而，现有基准主要测量静态工具集下的实例级成功，对代理获取此类可重用技能的能力提供有限的见解。我们通过引入SkillCraft基准，专门测试代理形成和重用更高层次工具组合的能力，其中我们称之为技能。SkillCraft具有真实、高度组合化的工具使用场景，难度沿定量和结构维度扩展，旨在激发技能抽象和跨任务重用。我们进一步提出了一种轻量级的评估协议，使代理能够自动将基本工具组合成可执行的技能，缓存并重用它们在任务内和跨任务中，从而提高效率的同时积累可重用的技能库。在SkillCraft上评估最先进的代理，我们观察到显著的效率提升，通过技能节省和重用，token使用量减少了高达80%。此外，成功率与测试时的工具组合能力强相关，突显了组合技能获取作为核心能力。
英文摘要：
Real-world tool-using agents operate over long-horizon workflows with recurring structure and diverse demands, where effective behavior requires not only invoking atomic tools but also abstracting, and reusing higher-level tool compositions. However, existing benchmarks mainly measure instance-level success under static tool sets, offering limited insight into agents' ability to acquire such reusable skills. We address this gap by introducing SkillCraft, a benchmark explicitly stress-test agent ability to form and reuse higher-level tool compositions, where we call Skills. SkillCraft features realistic, highly compositional tool-use scenarios with difficulty scaled along both quantitative and structural dimensions, designed to elicit skill abstraction and cross-task reuse. We further propose a lightweight evaluation protocol that enables agents to auto-compose atomic tools into executable Skills, cache and reuse them inside and across tasks, thereby improving efficiency while accumulating a persistent library of reusable skills. Evaluating state-of-the-art agents on SkillCraft, we observe substantial efficiency gains, with token usage reduced by up to 80% by skill saving and reuse. Moreover, success rate strongly correlates with tool composition ability at test time, underscoring compositional skill acquisition as a core capability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
