SkillSandbox：通过动态场景合成进行技能验证

中文摘要 AI 辅助

自我进化的智能体将任务解决经验提炼为技能以供未来复用，但这些技能可能编码了错误的流程或不可迁移的知识。因此，验证每个技能的可复用性至关重要：即其指导在超出其提炼来源的经验之外是否仍然有用。这种验证需要观察技能在新任务执行中的影响，然而现有任务可能无法暴露目标技能实际可被运用的情境。为了构建此类情境，我们提出了SkillSandbox，一个为每个技能动态合成任务及其环境的框架，这些任务与环境与技能相关但具有新颖性。一个提议者（Proposer）指定需保留的条件和需变化的源特定细节，一个构建者（Builder）构建可执行的场景，一个验证者（Verifier）比较有技能和无技能的执行情况。验证者评估可执行性、效用和效率，以给出保留（Keep）或拒绝（Reject）的判定，决定该技能是否进入技能库。在ALFWorld和WebShop上使用三种模型，SkillSandbox始终产生最强的下游性能和更高的执行效率。进一步的分析考察了这些收益是否反映了对技能可复用性的准确评估，并识别了SkillSandbox中哪些组件对其有所贡献。

英文摘要

Self-evolving agents distill task-solving experience into skills for future reuse, but these skills can encode incorrect procedures or non-transferable knowledge. It is therefore critical to verify each skill's reusability: whether its guidance remains useful beyond the experience from which it was distilled. Such verification requires observing how a skill affects execution in new tasks, yet existing tasks may not expose the situations where the target skill can actually be exercised. To construct such situations, we propose SkillSandbox, a framework that dynamically synthesizes a task and its environment for each skill that are skill-relevant yet novel. A Proposer specifies the conditions to preserve and the source-specific details to vary, a Builder constructs an executable scenario, and a Verifier compares executions with and without the skill. The Verifier assesses executability, utility, and efficiency to assign a Keep or Reject verdict, determining whether the skill enters the library. Across ALFWorld and WebShop with three models, SkillSandbox consistently yields the strongest downstream performance and improved execution efficiency. Further analyses examine whether these gains reflect accurate assessment of skill reusability and identify which components of SkillSandbox contribute to them.

发表机构

• Yonsei University（延世大学）

机构由 AI 辅助整理，请以论文原文为准。