arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.07153cs.AI
收藏

ANCHOR:用于GUI代理的分支点数据生成

ANCHOR: Branch-Point Data Generation for GUI Agents

Yale University(耶鲁大学);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校)

Jinbiao Wei, Yilun Zhao, Kangqi Ni, Arman Cohan

AI总结:

ANCHOR通过分支点生成高质量交互数据,提升桌面环境GUI代理的训练效果和泛化能力。

摘要:

端到端的桌面环境GUI代理需要大量高质量的交互数据,但收集人类演示成本高,现有合成流程常存在任务多样性有限或轨迹噪声大、目标漂移的问题。我们提出了ANCHOR轨迹扩展框架,从少量验证的种子演示出发,识别对应有意义的状态变化点,提出新的基于当前GUI上下文的任务变体。执行代理根据提出的指令生成新轨迹,而验证器通过状态感知检查和轨迹一致性确保任务完成。为进一步提升监督质量,我们还应用任务条件的步骤级过滤以移除不相关的动作,并对分支后段进行去噪以保持意图一致。在标准桌面基准OSWorld和WindowsAgentArena上的实验表明,基于我们扩展语料库微调的模型在零样本代理和代表性合成基线上的表现均有显著提升,并能跨应用和操作系统泛化。

英文摘要:

End-to-end GUI agents for real desktop environments require large amounts of high-quality interaction data, yet collecting human demonstrations is expensive and existing synthetic pipelines often suffer from limited task diversity or noisy, goal-drifting trajectories. We present a trajectory expansion framework Anchor that bootstraps scalable desktop supervision from a small set of verified seed demonstrations. Starting from each seed, we identify branch points that correspond to meaningful state changes and propose new, state-grounded task variants conditioned on the current GUI context. An executing agent then follows the proposed instructions to generate new trajectories, while a verifier enforces task completion via state-aware checks and trajectory-level consistency. To improve supervision quality, we further apply task-conditioned step-level filtering to remove ungrounded actions and denoise post-branch segments to maintain coherent intent. Experiments on standard desktop benchmarks, OSWorld and WindowsAgentArena, show that models fine-tuned on our expanded corpus achieve consistent improvements over zero-shot agents and representative synthesis baselines, and generalize across applications and operating systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑