arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.08049cs.AIcs.MA
收藏

SKILL.nb:用于持久代理工作流的选择性形式化与门控执行

SKILL.nb: Selective Formalization and Gated Execution for Durable Agent Workflows

ServiceNow Research;Mila;Polytechnique Montréal(蒙特利尔综合理工学院);Canada CIFAR AI Chair(加拿大CIFAR人工智能讲席)

Amine El Hattami, Nicolas Chapados, Christopher Pal

AI总结:

提出SKILL.nb框架,通过选择性形式化和门控执行管理代理工作流的生命周期可靠性,在WebArena-Verified上单轮成功率达53.7%,重执行保留率91.7%。

摘要:

AI代理越来越多地将过去的经验转化为可重用的工件,如代码、工作流和程序记忆。重用可以提高效率,但也带来了生命周期可靠性问题:曾经成功的工件可能在环境漂移、任务说明不充分或任务分布变化时失败,尤其是在Web自动化中。我们引入了SKILL.nb,一个通过证据校准的生命周期策略来管理可重用代理工作流的框架。SKILL.nb使用选择性形式化:执行证据决定哪些工作流步骤应成为可执行代码,哪些应保留自然语言指导,以及何时应修订这些选择。工作流存储为可审计、版本化的笔记本,交织自然语言指导、多语言可执行单元格、验证门、回退路径以及多模态证据(如输出、截图和错误轨迹)。在运行时,门控执行让每个步骤在门验证时运行代码,或在漂移使可执行实现失效时本地回退。在WebArena-Verified上,SKILL.nb实现了53.7%的单轮成功率,比最强基线提高了3.9个百分点。在三次重新执行中,它保留了91.7%的初始成功任务,比次优方法高出15.5个百分点。在有界修复下,它恢复了72.9%的后续失败,同时将修复后回归限制在4.2%,而持久基线为15.0%至17.0%。它还在Mind2Web跨网站和跨领域分割上领先。在GitLab迁移测试中,SKILL.nb在重用基于GitLab 15.7学习的冻结状态时保持性能,冻结与新鲜目标版本的差距在GitLab 16.11上为-1.7个百分点,在GitLab 18.9上为+0.6个百分点。这些结果将生命周期治理和门控执行确定为超越一次性任务成功之外的可靠性轴。

英文摘要:

AI agents increasingly turn past experience into reusable artifacts such as code, workflows, and procedural memories. Reuse can improve efficiency, but it also creates a lifecycle reliability problem: artifacts that succeed once may fail under environment drift, underspecified tasks, or changing task distributions, especially in web automation. We introduce SKILL.nb, a framework for governing reusable agent workflows with evidence-calibrated lifecycle policies. SKILL.nb uses selective formalization: execution evidence decides which workflow steps should become executable code, which should remain natural-language guided, and when those choices should be revised. Workflows are stored as auditable, versioned notebooks that interleave natural-language guidance, multi-language executable cells, validation gates, fallback paths, and multimodal evidence such as outputs, screenshots, and error traces. At runtime, gate-conditioned execution lets each step run code when its gates validate, or fall back locally when drift invalidates the executable realization. On WebArena-Verified, SKILL.nb achieves 53.7% single-round success, improving over the strongest baseline by 3.9 percentage points. Across three re-executions, it retains 91.7% of initially successful tasks, 15.5 points above the next best method. Under bounded repair, it recovers 72.9% of subsequent failures while limiting post-repair regressions to 4.2%, compared with 15.0% to 17.0% for persistent baselines. It also leads on Mind2Web cross-website and cross-domain splits. In a GitLab migration test, SKILL.nb preserves performance when reusing frozen state learned on GitLab 15.7, with frozen-versus-fresh target-version gaps of -1.7 points on GitLab 16.11 and +0.6 points on GitLab 18.9. These results identify lifecycle governance and gate-conditioned execution as reliability axes beyond one-shot task success.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑