arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.08533cs.CV
收藏

SecAgent:基于语义上下文的高效移动GUI代理

SecAgent: Efficient Mobile GUI Agent with Semantic Context

Taobao & Tmall Group of Alibaba(阿里巴巴淘天集团)

Yiping Xie, Song Chen, Jingxuan Xing, Wei Jiang, Zekun Zhu, Yingyao Wang, Pi Bu, Jun Song, Yuning Jiang, Bo Zheng

AI总结:

SecAgent通过构建中文移动GUI数据集和语义上下文机制,提升移动GUI代理的效率与性能,实现高效任务处理与多语言支持。

摘要:

SecAgent通过构建中文移动GUI数据集和语义上下文机制,提升移动GUI代理的效率与性能,实现高效任务处理与多语言支持。

英文摘要:

Mobile Graphical User Interface (GUI) agents powered by multimodal large language models have demonstrated promising capabilities in automating complex smartphone tasks. However, existing approaches face two critical limitations: the scarcity of high-quality multilingual datasets, particularly for non-English ecosystems, and inefficient history representation methods. To address these challenges, we present SecAgent, an efficient mobile GUI agent at 3B scale. We first construct a human-verified Chinese mobile GUI dataset with 18k grounding samples and 121k navigation steps across 44 applications, along with a Chinese navigation benchmark featuring multi-choice action annotations. Building upon this dataset, we propose a semantic context mechanism that distills history screenshots and actions into concise, natural language summaries, significantly reducing computational costs while preserving task-relevant information. Through supervised and reinforcement fine-tuning, SecAgent outperforms similar-scale baselines and achieves performance comparable to 7B-8B models on our and public navigation benchmarks. Our dataset is available at https://huggingface.co/datasets/alibabagroup/CMGUI.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑