Recent advances in Multimodal Large Language Models (MLLMs) have substantially driven the progress of autonomous agents for Graphical User Interface (GUI). Nevertheless, in real-world applications, GUI agents are often faced with non-stationary environments, leading to high computational costs for data curation and policy optimization. In this report, we introduce a novel MLLM-centered framework for GUI agents, which consists of two components: agentic-Q estimation and step-wise policy optimization. The former one aims to optimize a Q-model that can generate step-wise values to evaluate the contribution of a given action to task completion. The latter one takes step-wise samples from the state-action trajectory as inputs, and optimizes the policy via reinforcement learning with our agentic-Q model. It should be noticed that (i) all state-action trajectories are produced by the policy itself, so that the data collection costs are manageable; (ii) the policy update is decoupled from the environment, ensuring stable and efficient optimization. Empirical evaluations show that our framework endows Ovis2.5-9B with powerful GUI interaction capabilities, achieving remarkable performances on GUI navigation and grounding benchmarks and even surpassing contenders with larger scales.
Empathic and agentic artificial intelligence in nursing: perspectives on a human-centered framework for cancer care navigation in the United States
University of Tennessee Health Science Center, College of Nursing(田纳西大学健康科学中心护理学院);University of Tennessee Health Science Center, Center for Biomedical Informatics, Department of Pediatrics(田纳西大学健康科学中心生物医学信息中心,儿科系);University of Tennessee Health Science Center, Department of Radiation Oncology(田纳西大学健康科学中心放射肿瘤学系)
PreAct-Nav: Agentic Reasoning Before Action for Urban Navigation
Shanghai Jiao Tong University(上海交通大学);Beihang University(北京航空航天大学);The Hong Kong University of Science and Technology(香港科技大学);Tsinghua University(清华大学);University of Cambridge(剑桥大学);Intelligent Game and Decision Lab (IGDL)(智能博弈与决策实验室)
MANA: Towards Efficient Mobile Ad Detection via Multimodal Agentic UI Navigation
Department of Computer Science and Technology(计算机科学与技术系);University of Southern California(南加州大学);Shanghai Jiao Tong University(上海交通大学);State Key Laboratory of Internet Architecture(互联网架构国家重点实验室)
Accelerating Manufacturing Scale-Up from Material Discovery Using Agentic Web Navigation and Retrieval-Augmented AI for Process Engineering Schematics Design
RALLY: Role-Adaptive LLM-Driven Yoked Navigation for Agentic UAV Swarms
Hangzhou Institute for Advanced Study, UCAS(杭州高等研究机构,中国科学院大学);Zhejiang University(浙江大学);Zhejiang Lab(浙江实验室);Macau University of Science and Technology(澳门科学理工学院)
GUI Exploration Lab: Enhancing Screen Navigation in Agents via Multi-Turn Reinforcement Learning
Beijing University of Posts and Telecommunications(北京邮电大学);StepFun;Waseda University(早稻田大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所)
A Practical Guide for Designing, Developing, and Deploying Production-Grade Agentic AI Workflows
Old Dominion University(旧 Dominion 大学);Florida International University(佛罗里达国际大学);Nanyang Technological University(南洋理工大学);University of Colombo(科伦坡大学)
LLM-Guided Indoor Navigation with Multimodal Map Understanding
Institute of Information Science and Technologies (ISTI)(信息科学与技术研究所);National Research Council of Italy (CNR)(意大利国家研究理事会);Department of Computer Science(计算机科学系);University of Pisa(比萨大学);University College Dublin (UCD)(都柏林大学学院);School of Computer Science(计算机科学学院)
Navigation is an essential ability for mobile agents to be completely autonomous and able to perform complex actions. However, the problem of navigation for agents with limited (or no) perception of the world, or devoid of a fully defined motion model, has received little attention from research in AI and Robotics. One way to tackle this problem is to use guided navigation, in which other autonomous agents, endowed with perception, can combine their distinct viewpoints to infer the localisation and the appropriate commands to guide a sensory deprived agent through a particular path. Due to the…
A Practical Guide to Agentic AI Transition in Organizations
Old Dominion University(旧 Dominion 大学);Florida International University(佛罗里达国际大学);Nanyang Technological University(南洋理工大学);University of Colombo(科伦坡大学);Center for Wireless Communications, University of Oulu(无线通信中心,奥卢大学);University of Sri Jayewardenepura(斯里贾yenepura大学);Accenture Technology Labs(埃森哲技术实验室)