World models have shown great utility in improving the task performance of embodied agents. While prior work largely focuses on pixel-space world models, these approaches face practical limitations in GUI settings, where predicting complex visual elements in future states is often difficult. In this work, we explore an alternative formulation of world modeling for GUI agents, where state transitions are described in natural language rather than predicting raw pixels. First, we introduce MobileWorldBench, a benchmark that evaluates the ability of vision-language models (VLMs) to function as world models for mobile GUI agents. Second, we release MobileWorld, a large-scale dataset consisting of 1.4M samples, that significantly improves the world modeling capabilities of VLMs. Finally, we propose a novel framework that integrates VLM world models into the planning framework of mobile agents, demonstrating that semantic world models can directly benefit mobile agents by improving task success rates. The code and dataset is available at https://github.com/jacklishufan/MobileWorld
Nanyang Technological University(南洋理工大学);MiLM Plus, Xiaomi Inc.(小米公司);Independent Researchers(独立研究人员);Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学人工智能学院);Wuhan University(武汉大学);Xiamen University(厦门大学)
PowerLens: Taming LLM Agents for Safe and Personalized Mobile Power Management
China University of Geosciences (Beijing)(中国地质大学(北京));Shenzhen University(深圳大学);Hong Kong University of Science and Technology(香港科学与技术大学);City University of Hong Kong(香港城市大学)
Mobile-Seed: Joint Semantic Segmentation and Boundary Detection for Mobile Robots
Wuhan University(武汉大学);Technical University of Munich(慕尼黑工业大学);Nanyang Technological University(南洋理工大学);Institute of Infocomm Research (I2R)(资讯通信研究院);National University of Defense Technology(国防科技大学)
Federated Mixture-of-Experts Alignment on Mobile Edge Networks under Data Heterogeneity
Hong Kong JC STEM Lab of Smart City(香港JC STEM实验室);Department of Computer Science, City University of Hong Kong(香港城市大学计算机科学系);School of Computer Science and Technology, Xidian University(西安电子科技大学计算机科学与技术学院);Department of Electrical and Computer Engineering, The University of Hong Kong(香港大学电子与计算机工程系)
Semantic Enrichment of Mobile Phone Data Records Using Background Knowledge
University of Trento(特伦托大学);Massachusetts Institute of Technology, MIT(麻省理工学院);Fondazione Bruno Kessler(布鲁诺·凯斯勒基金会);SKIL LAB – Telecom Italia(意大利电信SKIL实验室)
Optimization and Mobile Deployment for Anthropocene Neural Style Transfer
Institute of Applied Arts\ Yang Ming Chiao Tung University Hsinchu City Taiwan (R.O.C);Institute of Applied Arts\ Yang Ming Chiao Tung University Hsinchu City Taiwan(R.O.C);Institute of Applied Arts\ Yang Ming Chiao Tung University