arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.16855cs.AIcs.CL
收藏

Mobile-Agent-v3.5: 多平台基础图形用户界面代理

Mobile-Agent-v3.5: Multi-platform Fundamental GUI Agents

Haiyang Xu, Xi Zhang, Haowei Liu, Junyang Wang, Zhaozai Zhu, Shengjie Zhou, Xuhao Hu, Feiyu Gao, Junjie Cao, Zihua Wang, Zhiyuan Chen, Jitong Liao, Qi Zheng, Ji… 展开作者

Haiyang Xu, Xi Zhang, Haowei Liu, Junyang Wang, Zhaozai Zhu, Shengjie Zhou, Xuhao Hu, Feiyu Gao, Junjie Cao, Zihua Wang, Zhiyuan Chen, Jitong Liao, Qi Zheng, Jiahui Zeng, Ze Xu, Shuai Bai, Junyang Lin, Jingren Zhou, Ming Yan

AI总结:

GUI-Owl-1.5通过多平台支持和创新算法在GUI任务中取得突破性成绩。

摘要:

本文介绍了GUI-Owl-1.5,最新的原生GUI代理模型,其具备多种大小(2B/4B/8B/32B/235B)的指令/思考变体,并支持多种平台(桌面、移动、浏览器等),以实现云边协作和实时交互。GUI-Owl-1.5在超过20个开源模型的GUI基准测试中取得了最先进的结果:(1)在GUI自动化任务中,其在OSWorld上获得56.5分,在AndroidWorld上获得71.�分,在WebArena上获得48.4分;(2)在定位任务中,其在ScreenSpotPro上获得80.3分;(3)在工具调用任务中,其在OSWorld-MCP上获得47.6分,在MobileWorld上获得46.8分;(4)在记忆和知识任务中,其在GUI-Knowledge Bench上获得75.5分。GUI-Owl-1.5集成了几个关键创新:(1)混合数据飞轮:我们基于模拟环境和基于云的沙盒环境构建了UI理解和轨迹生成的数据管道,以提高数据收集的效率和质量。(2)统一增强代理能力:我们使用统一的思维综合管道来增强模型的推理能力,特别强调提高关键代理能力,包括工具/MCP使用、记忆和多代理适应。(3)多平台环境RL扩展:我们提出了一种新的环境RL算法MRPO,以解决多平台冲突和长周期任务的低训练效率问题。GUI-Owl-1.5模型已开源,一个在线云沙盒演示可在https://github.com/X-PLUG/MobileAgent上获得。

英文摘要:

The paper introduces GUI-Owl-1.5, the latest native GUI agent model that features instruct/thinking variants in multiple sizes (2B/4B/8B/32B/235B) and supports a range of platforms (desktop, mobile, browser, and more) to enable cloud-edge collaboration and real-time interaction. GUI-Owl-1.5 achieves state-of-the-art results on more than 20+ GUI benchmarks on open-source models: (1) on GUI automation tasks, it obtains 56.5 on OSWorld, 71.6 on AndroidWorld, and 48.4 on WebArena; (2) on grounding tasks, it obtains 80.3 on ScreenSpotPro; (3) on tool-calling tasks, it obtains 47.6 on OSWorld-MCP, and 46.8 on MobileWorld; (4) on memory and knowledge tasks, it obtains 75.5 on GUI-Knowledge Bench. GUI-Owl-1.5 incorporates several key innovations: (1) Hybird Data Flywheel: we construct the data pipeline for UI understanding and trajectory generation based on a combination of simulated environments and cloud-based sandbox environments, in order to improve the efficiency and quality of data collection. (2) Unified Enhancement of Agent Capabilities: we use a unified thought-synthesis pipeline to enhance the model's reasoning capabilities, while placing particular emphasis on improving key agent abilities, including Tool/MCP use, memory and multi-agent adaptation; (3) Multi-platform Environment RL Scaling: We propose a new environment RL algorithm, MRPO, to address the challenges of multi-platform conflicts and the low training efficiency of long-horizon tasks. The GUI-Owl-1.5 models are open-sourced, and an online cloud-sandbox demo is available at https://github.com/X-PLUG/MobileAgent.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑