Recent progress in GUI agents has substantially improved visual grounding, yet robust planning remains challenging, particularly when the environment deviates from a canonical initial state. In real applications, users often invoke assistance mid-workflow, where software may be partially configured, steps may have been executed in different orders, or the interface may differ from its default setup. Such task-state variability is pervasive but insufficiently evaluated in existing GUI benchmarks. To address this gap, we introduce WorldGUI, a benchmark covering ten widely used desktop and web applications with tasks instantiated under diverse, systematically constructed initial states. These variations capture realistic human-computer interaction settings and enable diagnostic evaluation of an agent's ability to recover, adapt plans, and handle non-default contexts. We further present WorldGUI-Agent, a simple and model-agnostic framework that organizes planning and execution around three critique stages, improving reliability in dynamic environments. Experiments demonstrate that state-of-the-art GUI agents exhibit substantial performance degradation under non-default initial conditions, revealing limited robustness and fragile planning behaviors. Our benchmark and framework provide a foundation for developing more adaptable and reliable GUI agents. The code and data are available at https://github.com/showlab/WorldGUI.
Beyond Sequential Interaction: Benchmarking Parallel Execution and Coordination for GUI Agents
Beijing University of Posts and Telecommunications(北京邮电大学);State Key Laboratory for General Artificial Intelligence, BIGAI(通用人工智能国家重点实验室,字节跳动公司人工智能研究院);China University of Geosciences (Beijing)(中国地质大学(北京));Beijing Institute of Technology(北京理工大学)
OS-Oracle: A Comprehensive Framework for Cross-Platform GUI Critic Models
Shanghai Jiaotong University(上海交通大学);Shanghai AI Laboratory(上海人工智能实验室);CUHK MMLab(香港大学MMLab);The University of Hong Kong(香港大学);The Hong Kong University of Science and Technology(香港科学与技术大学)
GUI-World: A Video Benchmark and Dataset for Multimodal GUI-oriented Understanding
Huazhong University of Science and Technology(华中科技大学);University of Notre Dame(圣母大学);Imperial College London(伦敦帝国学院);Peking University(北京大学);Microsoft Research(微软研究院);Lehigh University(理海大学)
CADWorld: Computer-Use Benchmark for Long-Horizon Computer-Aided Design
Georgia Institute of Technology(佐治亚理工学院);North Carolina State University(北卡罗来纳州立大学);Marquette University(马凯特大学);Saros Lab(Saros实验室);CAMEL-AI;National University of Singapore(新加坡国立大学)
OS-Helix: Unifying GUI Interaction and Code Execution in Native Computer-Using Agents
University of Science and Technology of China(中国科学技术大学);Shanghai AI Laboratory(上海人工智能实验室);National University of Singapore(新加坡国立大学);Shanghai Jiao Tong University(上海交通大学);Fudan University(复旦大学);Nanjing University(南京大学);Xi’an Jiaotong University(西安交通大学);The University of Hong Kong(香港大学)
MacAgentBench: Benchmarking AI Agents on Real-World macOS Desktop
Shanghai Jiao Tong University(上海交通大学);Shanghai AI Laboratory(上海人工智能实验室);Xi’an Jiaotong University(西安交通大学);Fudan University(复旦大学);University of Science and Technology of China(中国科学技术大学);Zhejiang University(浙江大学);East China Normal University(华东师范大学);Tsinghua University(清华大学)
提出MacAgentBench,包含676个跨25个应用的任务,采用规则评估和细粒度多检查点评分,实验表明最佳配置(OpenClaw上的Claude Opus 4.6)达到73.7% Pass@1,优势主要来自技能库而非框架设计。
UI-MOPD: Multi-Platform On-Policy Distillation for Unified GUI Agents
Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Xiaomi(小米);Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));Zhejiang University(浙江大学);Peng Cheng Laboratory(鹏城实验室)
ScaleWoB: Guiding GUI Agents with Coding Agents via Large-Scale Environmental Synthesis
Institute for AI Industry Research (AIR), Tsinghua University(人工智能产业研究院(AIR),清华大学);University of Electronic Science and Technology of China(电子科技大学);MiLM Plus, Xiaomi Inc.(小米公司MiLM Plus团队)
PAGER: Bridging the Semantic-Execution Gap in Point-Precise Geometric GUI Control
University of Chinese Academy of Sciences(中国科学院大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);China University of Petroleum-Beijing(中国石油大学(北京))
VisualTrap: A Stealthy Backdoor Attack on GUI Agents via Visual Grounding Manipulation
University of Science and Technology of China(中国科学技术大学);National University of Singapore(新加坡国立大学);East China University of Science and Technology(东华大学)