Mobile Graphical User Interface (GUI) agents aim to autonomously complete tasks within or across apps based on user instructions. While recent Multimodal Large Language Models (MLLMs) enable these agents to interpret UI screens and perform actions, existing agents remain fundamentally reactive. They reason over the current UI screen but lack a structured representation of the app navigation flow, limiting GUI agents' ability to understand execution context, detect unexpected execution results, and recover from errors. We introduce Agent-SAMA, a state-aware multi-agent framework that models app execution as a Finite State Machine (FSM), treating UI screens as states and user actions as transitions. Agent-SAMA implements four specialized agents that collaboratively construct and use FSMs in real time to guide task planning, execution verification, and recovery. We evaluate Agent-SAMA on two types of benchmarks: cross-app (Mobile-Eval-E, SPA-Bench) and mostly single-app (AndroidWorld). On Mobile-Eval-E, Agent-SAMA achieves an 84.0% success rate and a 71.9% recovery rate. On SPA-Bench, it reaches an 80.0% success rate with a 66.7% recovery rate. Compared to prior methods, Agent-SAMA improves task success by up to 12% and recovery success by 13.8%. On AndroidWorld, Agent-SAMA achieves a 63.7% success rate, outperforming the baselines. Our results demonstrate that structured state modeling enhances robustness and can serve as a lightweight, model-agnostic memory layer for future GUI agents.
When Large Language Model Agents Meet 6G Networks: Perception, Grounding, and Alignment
Nanyang Technological University(南洋理工大学);School of Computer Science and Engineering(计算机科学与工程学院);Guangdong University of Technology(广东工业大学);School of Automation(自动化学院);Singapore University of Technology and Design(新加坡科技设计大学);Auburn University(奥本大学);University of Houston(休斯顿大学);Kyung Hee University(庆熙大学);Sungkyunkwan University(成均馆大学);Hong Kong University of Science and Technology(香港科技大学)
SafeMobile: Chain-level Jailbreak Detection and Automated Evaluation for Multimodal Mobile Agents
Nanyang Technological University(南洋理工大学);Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所);Beihang University(北京航空航天大学);Sun Yat-sen University(中山大学);Beijing Institute of Technology(北京理工大学);National University of Singapore(新加坡国立大学)
Align-KD: Distilling Cross-Modal Alignment Knowledge for Mobile Vision-Language Model Enhancement
State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University, China(通用人工智能国家重点实验室,智能科学与技术学院,北京大学,中国);Huawei Noah’s Ark Lab, China(华为诺亚方舟实验室,中国)