SkillMaster：迈向LLM代理自主技能掌握
SkillMaster: Toward Autonomous Skill Mastery in LLM Agents
AI总结：
SkillMaster通过轨迹引导技能复习、反事实效用评估和DualAdv-GRPO算法，使LLM代理能自主创建、优化和选择技能，提升任务成功率8.8%和9.3%。
摘要：
技能为提升LLM代理在复杂任务上的表现提供了有效机制，但现有代理框架中，其创建、优化和选择通常由外部教师、人工设计的规则或辅助模块控制。因此，技能仍被视为外部资源被调用，而非代理能够通过经验发展、适应和内化的能力。为赋予LLM代理自主技能掌握能力，我们提出了SkillMaster，一种训练框架，教导代理在任务解决过程中创建新技能、优化现有技能并选择积累的技能。这一能力通过三个关键设计实现。首先，我们通过轨迹引导的技能复习训练代理，教导代理根据已完成回合的证据提出、更新或保留技能。其次，每个候选技能编辑均设计为通过其反事实效用在相关探测任务上进行评估，为训练技能编辑决策提供直接学习信号。第三，我们引入了DualAdv-GRPO，分别估计任务解决动作和技能编辑决策的优势，稳定了任务解决与技能管理的联合训练。在ALFWorld和WebShop上的实验表明，SkillMaster相比最先进的基线方法将整体成功率提高了8.8%和9.3%，在所有比较方法中取得了最佳性能。进一步分析显示，受SkillMaster训练的代理能力发生了显著变化：代理能够识别技能失败，从轨迹证据中优化程序知识，并在有限的技能库编辑下将改进转移到未来任务中。总体而言，SkillMaster使LLM代理超越了单纯使用技能的阶段，成为能够发展、适应和应用自身技能库的自改进代理。
英文摘要：
Skills provide an effective mechanism for improving LLM agents on complex tasks, yet in existing agent frameworks, their creation, refinement, and selection are typically governed by external teachers, hand-designed rules, or auxiliary modules. As a result, skills remain external resources to be invoked, rather than capabilities that agents can develop, adapt, and internalize through experience. To endow LLM agents with autonomous skill mastery, we propose SkillMaster, a training framework that teaches agents to create new skills, refine existing skills, and select accumulated skills during task solving. This capability is achieved through three key designs. First, we train agents through trajectory-informed skill review, teaching agents to propose, update, or retain skills based on evidence from completed episodes. Second, each candidate skill edit is designed to be evaluated by its counterfactual utility on related probe tasks, providing a direct learning signal for training skill-editing decisions. Third, we introduce DualAdv-GRPO, which separately estimates advantages for task-solving actions and skill-editing decisions, stabilizing joint training across task solving and skill management. Experiments on ALFWorld and WebShop show that SkillMaster improves the overall success rate over state-of-the-art baselines by 8.8% and 9.3%, respectively, achieving the best performance among all compared methods. Further analysis reveals a marked shift in agent capability: agents trained with SkillMaster can identify skill failures, refine procedural knowledge from trajectory evidence, and transfer improvements to future tasks with limited skill-bank edits. Overall, SkillMaster moves LLM agents beyond mere skill use toward self-improving agents capable of developing, adapting, and applying their own skill repertoires.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
