Mobile Graphical User Interface (GUI) agents powered by multimodal large language models have demonstrated promising capabilities in automating complex smartphone tasks. However, existing approaches face two critical limitations: the scarcity of high-quality multilingual datasets, particularly for non-English ecosystems, and inefficient history representation methods. To address these challenges, we present SecAgent, an efficient mobile GUI agent at 3B scale. We first construct a human-verified Chinese mobile GUI dataset with 18k grounding samples and 121k navigation steps across 44 applications, along with a Chinese navigation benchmark featuring multi-choice action annotations. Building upon this dataset, we propose a semantic context mechanism that distills history screenshots and actions into concise, natural language summaries, significantly reducing computational costs while preserving task-relevant information. Through supervised and reinforcement fine-tuning, SecAgent outperforms similar-scale baselines and achieves performance comparable to 7B-8B models on our and public navigation benchmarks. Our dataset is available at https://huggingface.co/datasets/alibabagroup/CMGUI.
UI-Mem: Self-Evolving Experience Memory for Online Reinforcement Learning in Mobile GUI Agents
Multimedia Laboratory (MMLab), The Chinese University of Hong Kong(香港中文大学多媒体实验室);Shenzhen Loop Area Institute(深圳河套学院);Shanghai AI Lab(上海人工智能实验室);vivo AI Lab(vivo人工智能实验室);Princeton University(普林斯顿大学)
OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid Validation in Realistic Workflows
The University of Hong Kong(香港大学);Fudan University(复旦大学);Shanghai AI Laboratory(上海人工智能实验室);Nanyang Technological University(南洋理工大学);Nanjing University(南京大学);Shanghai Jiao Tong University(上海交通大学)
GUIOdyssey: A Comprehensive Dataset for Cross-App GUI Navigation on Mobile Devices
Shanghai AI Laboratory(上海人工智能实验室);The University of Hong Kong(香港大学);Nanjing University(南京大学);Shanghai Jiao Tong University(上海交通大学);Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳))
OpenMobile-2: Building Versatile Mobile Agents with Scalable Environments and App-Native Tools
Nanjing University(南京大学);Xi’an Jiaotong University(西安交通大学);National University of Singapore(新加坡国立大学);SenseTime(商汤科技);East China Normal University(华东师范大学);The University of Hong Kong(香港大学)