ClawTrace: 面向LLM智能体技能蒸馏的成本感知追踪
ClawTrace: Cost-Aware Tracing for LLM Agent Skill Distillation
AI总结：
针对技能蒸馏管道缺乏每步成本信号的问题，提出ClawTrace记录成本归因轨迹并生成TraceCard，通过CostCraft生成保留、剪枝和修复三类技能补丁，发现剪枝补丁作为质量护栏而保留补丁导致回归，主张按规则类型评估可复用技能。
摘要：
技能蒸馏管道从LLM智能体轨迹中学习可重用规则，但它们缺乏一个关键信号：每一步的成本。没有每步成本，管道无法区分添加缺失步骤以修复错误与移除从未影响结果的昂贵步骤。我们利用成本归因差距来探究蒸馏技能内部的规则类型是否以相同方式迁移到新任务。ClawTrace记录成本归因的智能体轨迹，并将每个会话编译成TraceCard；CostCraft读取TraceCard并编写三种技能补丁：保留、剪枝和修复。我们发现了一个聚合指标隐藏的模式。在30个保留的SpreadsheetBench任务上（两个种子），移除剪枝补丁大致使质量回归计数增加了三倍，而未降低中位成本。在整个84任务的SkillsBench迁移中，CostCraft未节省总成本。所有三个质量回归都追溯到保留通道，而两个质量提升都追溯到剪枝通道：剪枝补丁充当质量护栏，而保留补丁驱动回归。我们认为可重用的智能体技能应在规则类型层面进行评估，而不是作为整体指令包。为支持这一点，我们发布了ClawTrace、TraceCard模式以及全套类型化技能。
英文摘要：
Skill-distillation pipelines learn reusable rules from LLM agent trajectories, but they lack a key signal: how much each step costs. Without per-step cost, a pipeline cannot distinguish adding a missing step to fix a bug from removing an expensive step that never affected the outcome. We use the cost-attribution gap to ask whether the rule types inside a distilled skill transfer the same way to new tasks. ClawTrace records cost-attributed agent traces and compiles each session into a TraceCard; CostCraft reads TraceCards and writes three kinds of skill patches: preserve, prune, and repair. We find a pattern aggregate metrics hide. On 30 held-out SpreadsheetBench tasks across two seeds, removing prune patches roughly tripled the quality-regression count without lowering median cost. Across the full 84-task SkillsBench transfer, CostCraft saves no aggregate cost. All three quality regressions trace to the preserve lane, and both quality wins trace to the prune lane: prune patches act as quality guardrails while preserve patches drive regressions. We argue that reusable agent skills should be evaluated at the rule-type level, not as monolithic instruction packages. To support this, we release ClawTrace, the TraceCard schema, and the full set of typed skills.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
