The rapid advancement of vision-language models has catalyzed the emergence of GUI agents, which hold immense potential for automating complex tasks, from online shopping to flight booking, thereby alleviating the burden of repetitive digital workflows. As a foundational capability, GUI grounding is typically established as a prerequisite for end-to-end task execution. It enables models to precisely locate interface elements, such as text and icons, to perform accurate operations like clicking and typing. Unlike prior works that fine-tune models already possessing strong spatial awareness (e.g., Qwen3-VL), we aim to master the full technical pipeline by starting from a base model with minimal grounding ability, such as POINTS-1.5. We introduce POINTS-GUI-G-8B, which achieves state-of-the-art performance with scores of 59.9 on ScreenSpot-Pro, 66.0 on OSWorld-G, 95.7 on ScreenSpot-v2, and 49.9 on UI-Vision. Our model's success is driven by three key factors: (1) Refined Data Engineering, involving the unification of diverse open-source datasets format alongside sophisticated strategies for augmentation, filtering, and difficulty grading; (2) Improved Training Strategies, including continuous fine-tuning of the vision encoder to enhance perceptual accuracy and maintaining resolution consistency between training and inference; and (3) Reinforcement Learning (RL) with Verifiable Rewards. While RL is traditionally used to bolster reasoning, we demonstrate that it significantly improves precision in the perception-intensive GUI grounding task. Furthermore, GUI grounding provides a natural advantage for RL, as rewards are easily verifiable and highly accurate.
GoClick: Lightweight Element Grounding Model for Autonomous GUI Interaction
1 University of Chinese Academy of Sciences, Beijing, China. 2 New Laboratory of Pattern Recognition, State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China. 3 Hong Kong Institute of Science \& Innovation, Chinese Academy of Sciences, Hong Kong, China. Zhaoxiang Zhang
MedSPOT: A Workflow-Aware Sequential Grounding Benchmark for Clinical GUI
Gaash Research Lab, National Institute of Technology Srinagar, India(加什研究实验室,印度锡里纳杰尔国家理工学院);e\& Group, UAE(e&集团,阿联酋);Mohammad Bin Zayed University of Artificial Intelligence (MBZUAI), UAE(穆罕默德·本·扎耶德人工智能大学(MBZUAI),阿联酋);King Abdullah University of Science and Technology (KAUST), Saudi Arabia(国王 Abdullah 科学技术大学(KAUST),沙特阿拉伯)
How Auxiliary Reasoning Unleashes GUI Grounding in VLMs
Zhejiang Lab(浙江实验室);Hangzhou Research and Development Center(杭州研发中心);China Mobile(中国移动);Innovation Center of Yangtze River Delta(长江三角洲创新中心);Zhejiang University(浙江大学)
LSVG: Language-Guided Scene Graphs with 2D-Assisted Multi-Modal Encoding for 3D Visual Grounding
School of Automation Science and Engineering, South China University of Technology(自动化科学与工程学院,华南理工大学);School of Future Technology, South China University of Technology(未来技术学院,华南理工大学)
What Happens Before Decoding? Prefill Determines GUI Grounding in VLMs
Guangming Laboratory(光明实验室);National University of Singapore(新加坡国立大学);Peking University(北京大学);University of Waterloo(滑铁卢大学);The Chinese University of Hong Kong (Shenzhen)(香港中文大学(深圳))
Trifuse: Enhancing Attention-Based GUI Grounding via Multimodal Fusion
College of Computer Science and Technology, National University of Defense Technology(计算机科学与技术学院,国防科技大学);Intelligent Game and Decision Lab, Academy of Military Sciences(智能游戏与决策实验室,军事科学院)
Knowledge-guided Pairwise Reconstruction Network for Weakly Supervised Referring Expression Grounding
Institute of Computing Technology, CAS(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);University of Science and Technology of China(中国科学技术大学)
RGBX-R1: Visual Modality Chain-of-Thought Guided Reinforcement Learning for Multimodal Grounding
School of Artificial Intelligence, Tianjin University(天津大学人工智能学院);Low-Altitude Intelligence Lab, Xiong’an National Innovation Center(雄安国家创新中心低空智能实验室);Xiong’an Guochuang Lantian Technology Co., Ltd.(雄安国创莲田科技有限公司);College of Electronic Science and Technology, National University of Defense Technology(国防科技大学电子科学学院)
GUI-ARP: Enhancing Grounding with Adaptive Region Perception for GUI Agents
Wuhan University(武汉大学);Sun Yat-sen University(中山大学);Alibaba Group(阿里巴巴集团);East China Normal University(华东师范大学);University of Electronic Science and Technology of China(电子科技大学)
Enhancing Medical Visual Grounding via Knowledge-guided Spatial Prompts
School of Computer Science and Engineering, Nanjing University of Science and Technology(南京理工大学计算机科学与工程学院);School of Computer Science and Engineering, Southeast University(东南大学计算机科学与工程学院);Yong Yoo Lin School of Medicine, National University of Singapore(新加坡国立大学杨潞龄医学院);Institute of High-Performance Computing, Agency for Science, Technology and Research(新加坡科技研究局高性能计算研究所)
EARL: Towards a Unified Analysis-Guided Reinforcement Learning Framework for Egocentric Interaction Reasoning and Pixel Grounding
Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University, Hong Kong SAR(香港理工大学电子与电气工程系);Division of Emerging Interdisciplinary Areas (EMIA), The Hong Kong University of Science and Technology, Hong Kong SAR(香港理工大学新兴跨学科领域研究中心)
Generalist Scanner Meets Specialist Locator: A Synergistic Coarse-to-Fine Framework for Robust GUI Grounding
University of California, San Diego(加州大学圣地亚哥分校);ByteDance(字节跳动);University of California, Merced(加州大学默塞德分校);University of Southern California(南加州大学);University at Buffalo(布法罗大学);The University of Queensland(昆士兰大学)
RankGround: Efficient High-Resolution GUI Grounding via Lightweight Reranker-Guided Crop Selection
Shenzhen University of Advanced Technology(深圳理工大学);Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);Xiamen University(厦门大学)
Improving GUI Grounding with Explicit Position-to-Coordinate Mapping
ServiceNow;Mila - Quebec AI Institute(魁北克人工智能研究所);Université de Montréal(蒙特利尔大学);York University(约克大学);Polytechnique Montréal(蒙特利尔理工学院);McGill University(麦吉尔大学)