arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2512.14014cs.AI
收藏

MobileWorldBench: 向移动智能体的语义世界建模迈进

MobileWorldBench: Towards Semantic World Modeling For Mobile Agents

UCLA(加州大学洛杉矶分校);Panasonic AI Research(松下人工智能研究);Salesforce AI Research(Salesforce人工智能研究)

Shufan Li, Konstantinos Kallidromitis, Akash Gokul, Yusuke Kato, Kazuki Kozuka, Aditya Grover

AI总结:

MobileWorldBench通过语义世界模型提升移动智能体任务成功率

摘要:

世界模型在提高具身智能体任务性能方面表现出巨大实用性。尽管先前工作主要集中在像素空间的世界模型上,但这些方法在GUI设置中面临实际限制,因为在未来状态中预测复杂的视觉元素往往很困难。在本工作中,我们探索了一种针对GUI智能体的世界建模替代方法,其中状态转换用自然语言描述,而不是预测原始像素。首先,我们介绍了MobileWorldBench,这是一个评估视觉语言模型(VLMs)作为移动GUI智能体世界模型能力的基准。其次,我们发布了MobileWorld,一个包含140万样本的大型数据集,显著提升了VLMs的世界建模能力。最后,我们提出了一种新的框架,将VLM世界模型整合到移动智能体的规划框架中,证明语义世界模型可以通过提高任务成功率直接帮助移动智能体。代码和数据集可在https://github.com/jacklishufan/MobileWorld获取。

英文摘要:

World models have shown great utility in improving the task performance of embodied agents. While prior work largely focuses on pixel-space world models, these approaches face practical limitations in GUI settings, where predicting complex visual elements in future states is often difficult. In this work, we explore an alternative formulation of world modeling for GUI agents, where state transitions are described in natural language rather than predicting raw pixels. First, we introduce MobileWorldBench, a benchmark that evaluates the ability of vision-language models (VLMs) to function as world models for mobile GUI agents. Second, we release MobileWorld, a large-scale dataset consisting of 1.4M samples, that significantly improves the world modeling capabilities of VLMs. Finally, we propose a novel framework that integrates VLM world models into the planning framework of mobile agents, demonstrating that semantic world models can directly benefit mobile agents by improving task success rates. The code and dataset is available at https://github.com/jacklishufan/MobileWorld

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑