SkillForge: 在云技术支持中锻造领域特定的自我进化代理技能
SkillForge: Forging Domain-Specific, Self-Evolving Agent Skills in Cloud Technical Support
AI总结：
SkillForge通过闭环创建-评估-改进机制，解决云技术支持中领域特定技能缺乏和持续优化的问题，实验表明其能显著提升技能质量。
摘要：
在企业场景如云技术支持中部署基于LLM的代理需要高质量的领域特定技能。然而，现有技能创建者缺乏领域知识，生成的技能与现实任务需求不匹配。此外，一旦部署，没有系统机制将执行失败追溯到技能缺陷并驱动针对性改进，导致技能质量停滞。我们引入SkillForge，一个自我进化框架，闭合端到端的创建-评估-改进循环。为了生成良好的初始技能，领域情境化技能创建器在知识库和历史支持工单中接地技能合成。为了实现持续自我优化，三阶段流程——故障分析员、技能诊断员和技能优化器——自动批量诊断执行失败，定位底层技能缺陷，并重写技能以消除它们。此循环反复运行，使技能在每次部署反馈中自我改进。在五个真实世界云支持场景中评估，涵盖1,883张工单和3,737个任务，实验显示：(1) 领域情境化技能创建器生成的初始技能比通用技能创建器更好，以与历史工单中专家撰写的参考响应的一致性衡量；(2) 自我进化循环从多样起点（包括专家撰写的、领域创建的和通用技能）逐步改进技能质量，证明自动化进化可以超越手动编纂的专家知识。
英文摘要：
Deploying LLM-powered agents in enterprise scenarios such as cloud technical support demands high-quality, domain-specific skills. However, existing skill creators lack domain grounding, producing skills poorly aligned with real-world task requirements. Moreover, once deployed, there is no systematic mechanism to trace execution failures back to skill deficiencies and drive targeted refinements, leaving skill quality stagnant despite accumulating operational evidence. We introduce SkillForge, a self-evolving framework that closes an end-to-end creation-evaluation-refinement loop. To produce well-aligned initial skills, a Domain-Contextualized Skill Creator grounds skill synthesis in knowledge bases and historical support tickets. To enable continuous self-optimization, a three-stage pipeline -- Failure Analyzer, Skill Diagnostician, and Skill Optimizer -- automatically diagnoses execution failures in batch, pinpoints the underlying skill deficiencies, and rewrites the skill to eliminate them. This cycle runs iteratively, allowing skills to self-improve with every round of deployment feedback. Evaluated on five real-world cloud support scenarios spanning 1,883 tickets and 3,737 tasks, experiments show that: (1) the Domain-Contextualized Skill Creator produces substantially better initial skills than the generic skill creator, as measured by consistency with expert-authored reference responses from historical tickets; and (2) the self-evolution loop progressively improves skill quality from diverse starting points (including expert-authored, domain-created, and generic skills) across successive rounds, demonstrating that automated evolution can surpass manually curated expert knowledge.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
