发表机构
Zhejiang University; UESTC（浙江大学; 电子科技大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对GUI智能体静态技能失效问题，提出免训练框架EvoSkill-GUI，通过反思-修订-复用循环利用执行反馈动态修订技能包，在三大基准上无需训练即提升多个基础模型性能。
AI 中文摘要
GUI智能体在动态图形用户界面上执行长期任务，其中弹窗、延迟加载和重新定位的组件通常会使执行前固定的计划失效。最近的智能体技能框架通过封装可复用的程序性知识来缓解这一问题，然而现有技能设计大多未针对GUI执行动态性进行开发，并将技能视为部署前产生的静态产物，而非通过执行得以改进的活程序性知识。我们认为，GUI智能体需要的不是更好的静态技能，而是能够在部署时根据执行反馈进行修订、且无需额外训练的技能。我们提出EvoSkill-GUI，一个免训练框架，其中每个技能是一个结构化的多文件包，包含检索元数据、可执行计划、备份定位、失败恢复规则、可访问性工具和失败案例。EvoSkill-GUI通过“反思-修订-复用”循环运作：执行器在轨迹内进行即时修订，一个隔离的批评者在严格信息隔离下诊断失败轨迹，执行器通过受限工具接口编辑特定技能文件。在MobileWorld、AndroidWorld和OSWorld这三个涵盖移动和桌面平台的主流GUI基准上，EvoSkill-GUI在无需任何训练的情况下持续改进多个基础模型，最大提升分别为+16.2%、+6.0%和+10.5%，且进化后的技能库继续惠及相关任务，而非从头重建。我们的代码可在该https URL获取。
英文摘要
GUI agents execute long-horizon tasks on dynamic graphical user interfaces, where pop-ups, delayed loads, and relocated widgets routinely invalidate plans fixed before execution. Recent agent-skill frameworks encapsulate reusable procedural knowledge to mitigate this, yet existing skill designs are largely developed without targeting GUI execution dynamics and treat skills as static artifacts produced before deployment rather than living procedural knowledge that improves through it. We argue that what GUI agents need is not better static skills, but skills that can be revised from execution feedback at deployment time, without additional training. We propose \textbf{EvoSkill-GUI}, a training-free framework in which each skill is a structured multi-file package containing retrieval metadata, executable plans, backup localization, failure-recovery rules, accessibility utilities, and failure cases. EvoSkill-GUI operates through a \textbf{\emph{reflect-revise-reuse}} loop: the executor performs instant in-rollout revisions, an isolated critic diagnoses failed trajectories under strict information isolation, and the executor edits specific skill files through a restricted tool interface. Across MobileWorld, AndroidWorld, and OSWorld, three mainstream GUI benchmarks spanning mobile and desktop platforms, EvoSkill-GUI consistently improves multiple base models without any training, with maximum gains of $+16.2\%$, $+6.0\%$, and $+10.5\%$ respectively, and evolved skill libraries continue to benefit related tasks rather than being rebuilt from scratch. Our code is available at https://github.com/ZJU-REAL/EvoSkill-GUI.
CommentsProject Page: https://zju-real.github.io/EvoSkill-GUI/ Code: https://github.com/ZJU-REAL/EvoSkill-GUI
