发表机构
Beihang University; Shandong University（北京航空航天大学; 山东大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
SkillShapley是面向大语言模型智能体技能步骤归因的框架，将技能步骤归因建模为夏普利值估计问题，经SkillsBench实验可高效识别技能步骤价值，为智能体技能构建提供启示。
AI 中文摘要
智能体技能是关键的外部指令，使语言智能体能够执行编码、文档处理等长流程任务。现有智能体技能主要通过人工手动构建或智能体执行轨迹生成，对特定任务中各步骤对整体技能性能的贡献理解有限，即量化智能体技能内单个步骤的贡献仍是未解决的问题。为解决该问题，我们首先将技能步骤归因建模为基于夏普利值的贡献估计问题，进而提出SkillShapley——一种面向智能体技能的步骤级归因框架。值得注意的是，SkillShapley基于关键实证见解分为两个阶段运行，即离散化基准奖励会产生急剧性能突变，且步骤交互大多为加性而非协同性。具体而言，它首先识别信息性联盟区域，然后自适应采样新联盟以产生可复用的边际证据。在广泛采用的SkillsBench技能上开展的实验表明，SkillShapley可高效识别高价值或低价值技能步骤，为智能体技能构建提供若干关键启示。
英文摘要
Agent skills are crucial external instructions that enable language agents to execute long procedural tasks such as coding or document processing. Existing agent skills are primarily created through human manual crafting or agent execution traces, with limited understanding of how each step contributes to overall skill performance on specific tasks; i.e., there remains an open problem in quantifying the contribution of individual steps within an agent skill. To address this issue, we first model skill-step attribution as a Shapley value-based contribution estimation problem, and then propose SkillShapley, a step-level attribution framework for agent skills. Notably, SkillShapley operates in two phases, motivated by key empirical insights, i.e., discretized benchmark rewards that create sharp performance cliffs, and step interactions that are largely additive rather than synergistic. Specifically, it first identifies informative coalitional regions, and then adaptively samples new coalitions that can yield reusable marginal evidence. Experiments on skills from the widely adopted SkillsBench demonstrate that our SkillShapley can effectively and efficiently identify high- or low-value skill steps, providing several key takeaways for agent skill creation.
Comments15 pages, 4 figures
