Computer Use Agents (CUAs) can act through both atomic GUI actions, such as click and type, and high-level tool calls, such as API-based file operations, but this hybrid action space often leaves them uncertain about when to continue with GUI actions or switch to tools, leading to suboptimal execution paths. This difficulty stems from the scarcity of high-quality interleaved GUI-Tool trajectories, the cost and brittleness of collecting real tool trajectories, and the lack of trajectory-level supervision for GUI-Tool path selection. In this paper, we propose ToolCUA, an end-to-end agent designed to learn optimal GUI-Tool path selection through a staged training paradigm. We first introduce an Interleaved GUI-Tool Trajectory Scaling Pipeline that repurposes abundant static GUI trajectories and synthesizes a grounded tool library, enabling diverse GUI-Tool trajectories without manual engineering or real tool-trajectory collection. We then perform Tool-Bootstrapped GUI RFT, combining warmup SFT with single-turn RL to improve decisions at critical GUI-Tool switching points. Finally, we optimize ToolCUA with Online Agentic RL in a high-fidelity GUI-Tool environment, guided by a Tool-Efficient Path Reward that encourages appropriate tool use and shorter execution paths. Experiments on OSWorld-MCP show that ToolCUA achieves 46.85% accuracy, a relative improvement of approximately 66% over the baseline, establishing a new state of the art among models of comparable scale. It also improves by 3.9% over GUI-only settings, demonstrating effective GUI-Tool orchestration. The results further suggest that training in a hybrid action space is a promising paradigm for real-world digital agents. Open-sourced here: https://x-plug.github.io/ToolCUA/
Robust and Efficient Tool Orchestration via Layered Execution Structures with Reflective Correction
Department of Electrical Engineering and Computer Science, University of Kansas, Lawrence, KS, USA(电气工程与计算机科学系,堪萨斯大学);NEC Laboratories America, Princeton, NJ, USA(NEC美国实验室,新泽西州普林斯顿);Institute for Infocomm Research, A*STAR, Singapore(信息通信研究所,A*STAR,新加坡);School of Computer Science, University of Auckland, Auckland, New Zealand(计算机科学学院,奥克兰大学);Department of Statistics, University of Wisconsin--Madison, Madison, WI, USA(统计学系,威斯康星大学麦迪逊分校)
A Practical Guide for Designing, Developing, and Deploying Production-Grade Agentic AI Workflows
Old Dominion University(旧 Dominion 大学);Florida International University(佛罗里达国际大学);Nanyang Technological University(南洋理工大学);University of Colombo(科伦坡大学)
Hybrid Orchestration of Edge AI and Microservices via Graph-based Self-Imitation Learning
School of Computer Science, South-Central Minzu University(中央民族大学南中央学院);Key Laboratory of Cyber-Physical Fusion Intelligent Computing, State Ethnic Affairs Commission(国家民族事务委员会网络物理融合智能计算重点实验室);Hubei Key Laboratory of Smart Internet Technology, School of Electronic Information and Communication, Huazhong University of Science and Technology(湖北智能互联网技术重点实验室,华中科技大学电子信息与通信学院);Centre for Frontier AI Research, Agency for Science, Technology and Research (A*STAR)(前沿人工智能研究中心,科技研究局(A*STAR))
Towards Scalable Lightweight GUI Agents via Multi-role Orchestration
Zhejiang Key Laboratory of Accessible Perception and Intelligent Systems, Zhejiang University(浙江可及感知与智能系统重点实验室,浙江大学);College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);AntGroup(蚂蚁集团)
OrchJail: Jailbreaking Tool-Calling Text-to-Image Agents by Orchestration-Guided Fuzzing
Institute of Software, Chinese Academy of Sciences, Beijing, China;Science \& Technology on Integrated Information System Laboratory, Beijing, China;State Key Laboratory of Complex System Modeling;University of Chinese Academy of Sciences, Beijing, China
Tsinghua University(清华大学);Shanghai Jiao Tong University(上海交通大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Renmin University of China(中国人民大学)
Difficulty-Aware Agentic Orchestration for Query-Specific Multi-Agent Workflows
South China Normal University(华南师范大学);The University of Texas Health Science Center(德克萨斯大学健康科学中心);The Hong Kong University of Science(香港科学大学);University of California(加州大学);Peking University(北京大学);University of Toronto(多伦多大学)
Graph-Based Agentic AI with LangGraph: Workflow Pathways for Long-Running Stateful Business Processes
Dhillon School of Business, University of Lethbridge(莱斯布里奇大学迪隆商学院);Universidad de Guadalajara(瓜达拉哈拉大学);Al Hussein Technical University(侯赛因技术大学);Universidad de Guanajuato(瓜纳华托大学)
Benchmarking Multi-Agent LLM Architectures for Financial Document Processing: A Comparative Study of Orchestration Patterns, Cost-Accuracy Tradeoffs and Production Scaling Strategies
Department of Computer Science and Engineering(计算机科学与工程系);New York University(纽约大学);New York, NY 10012(纽约市NY 10012)
LLM-Based Data Science Agents: A Survey of Capabilities, Challenges, and Future Directions
York University(约克大学);Vector Institute for AI(人工智能矢量研究所);Dialpad Inc.(Dialpad公司);Nanyang Technological University(南洋理工大学);Salesforce AI Research(Salesforce人工智能研究)
Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows?
The University of Hong Kong(香港大学);Shanghai Jiao Tong University(上海交通大学);Google Cloud AI Research(谷歌云人工智能研究院);Google DeepMind(谷歌DeepMind);Salesforce Research(salesforce研究院);Yale University(耶鲁大学);Sea AI Lab(Sea人工智能实验室);University of Waterloo(滑铁卢大学)