技能问题：从优化编码智能体的仓库SKILL中汲取的经验教训
Skill Issue: Lessons from Optimizing Repository SKILLs for Coding Agents
- Technical University of Munich（慕尼黑工业大学）
- JetBrains Research（JetBrains 研究院）
- Constructor University Bremen（不来梅康斯特大学）
机构由 AI 辅助整理，请以论文原文为准。
中文总结 AI 辅助
针对编码智能体，通过挖掘合并拉取请求作为困难任务，比较GEPA与SkillOpt优化仓库SKILL文档的效果，发现GEPA提升4.9个百分点且文档蕴含项目特有知识。
中文摘要 AI 辅助
编码智能体越来越多地从SKILL中读取仓库知识——SKILL是与代码一起版本化的纯\ exttt{.md}文件。近期工作通过针对基准优化文档来自动合成这些文件。一个裸仓库没有附带基准，而先前工作构建的合成任务规模太小，以至于一个能力强的智能体在没有任何文档的情况下就能在这些任务上达到饱和。我们挖掘更困难的任务——仓库的合并拉取请求，在单个冻结的基础提交上回退；并通过同一智能体在有文档时是否比无文档时表现更好来对候选文档评分。在三个Kotlin仓库上，GEPA找到的文档平均将这一分数提高了4.9个百分点，而SkillOpt找到的文档则使其停留在起点，仅比种子高0.1个百分点。GEPA的提升与先前工作使用同一优化器报告的结果一致，并且在单个仓库所能提供的数据集规模下，该提升无法与智能体运行间的方差区分开来；要解决这一问题，需要的任务数量超过一个仓库历史所能提供的。这些文档本身比分数更具可读性：一个仓库的维护者从中发现了只有在该项目中工作才能获得的知识。
英文摘要
Coding agents increasingly read repository knowledge from SKILLs --- plain \texttt{.md} files versioned alongside the code. Recent work synthesizes these files automatically, by optimizing the document against a benchmark. A bare repository comes with no benchmark, and the synthetic tasks prior work builds are small enough that a capable agent saturates them with no document at all. We mine harder tasks --- merged pull requests of the repository, reverted at a single frozen base commit; and score a candidate document by whether the same agent does better with it than without it. On three Kotlin repositories, the documents GEPA finds raise this score by $4.9$pp on average, and the ones SkillOpt finds leave it where it started, $0.1$pp above the seed. The GEPA gain matches what prior work reports with the same optimizer, and at the dataset size a single repository supplies it cannot be separated from the agent's run-to-run variance; settling that would take more tasks than one repository's history yields. The documents themselves read better than the score: a maintainer of one repository found in them knowledge one only gets by working in the project.
