SkillEvolBench：从情景经验到程序性技能的演化基准测试
SkillEvolBench: Benchmarking the Evolution from Episodic Experience to Procedural Skills
AI总结：
提出SkillEvolBench基准，通过六个真实环境中的180个任务，评估大语言模型代理能否将情景经验提炼为可复用的程序性技能，发现当前代理难以形成稳健的技能，且原始轨迹复用优于蒸馏技能。
摘要：
大语言模型（LLM）代理在解决现实任务时会积累丰富的情景轨迹，但目前尚不清楚这种经验能否被提炼为可复用的程序性技能。我们引入了SkillEvolBench，一个用于评估从经验复用到技能形成这一步骤的诊断基准。它包含跨越六个真实代理环境的180个任务，组织成具有共享潜在程序的角色条件任务族。代理从获取任务中学习，使用压缩轨迹和验证器反馈更新外部技能库，然后面对冻结部署任务，测试上下文偏移、对抗性捷径和组合。通过将自生成和策划起始的技能演化与无技能和原始轨迹控制进行比较，SkillEvolBench将程序抽象与基础能力、策划的先验知识和情景轨迹的直接复用分离开来。在十种模型配置和三种代理框架下，我们发现当前代理通常能局部适应，但很少形成稳健的可复用技能。基于技能的条件可以改善获取或重放，个别模型有时在特定部署轴上有所提升，但这些提升在冻结部署下不稳定。原始轨迹复用经常优于蒸馏技能，表明当前的抽象过程丢弃了未来任务仍有用的上下文和程序线索。容量和成本分析进一步表明，编写更多技能或更大的三级资源库并不足够：额外的更新可以改善覆盖范围，同时引入特定于情节的漂移和程序混乱。这些发现将SkillEvolBench定位为一个测试平台，用于衡量一次性经验何时成为持久的程序性知识而非任务局部记忆。
英文摘要：
Large language model (LLM) agents accumulate rich episodic trajectories while solving real-world tasks, but it remains unclear whether such experience can be distilled into reusable procedural skills. We introduce SkillEvolBench, a diagnostic benchmark for evaluating this step from experience reuse to skill formation. It contains 180 tasks across six real-world agent environments, organized into role-conditioned task families with shared latent procedures. Agents learn from acquisition tasks, update an external skill library using compacted trajectories and verifier feedback, and then face frozen deployment tasks testing context shift, adversarial shortcuts, and composition. By comparing self-generated and curated-start skill evolution against no-skill and raw-trajectory controls, SkillEvolBench separates procedural abstraction from base capability, curated prior knowledge, and direct reuse of episodic traces. Across ten model configurations and three agent harnesses, we find that current agents often adapt locally but rarely form robust reusable skills. Skill-based conditions can improve acquisition or replay, and individual models sometimes gain on specific deployment axes, but these gains are unstable under frozen deployment. Raw-trajectory reuse frequently outperforms distilled skills, suggesting that current abstraction procedures discard contextual and procedural cues that remain useful for future tasks. Capacity and cost analyses further show that writing more skills or larger Tier-3 resource libraries is not sufficient: additional updates can improve coverage while introducing episode-specific drift and procedural clutter. These findings position SkillEvolBench as a testbed for measuring when one-off experience becomes durable procedural knowledge rather than task-local memory.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
