Building intelligent agents capable of dexterous manipulation is essential for achieving human-like automation in both robotics and digital environments. However, existing GUI agents rely on discrete click predictions (x,y), which prohibits free-form, closed-loop trajectories (e.g. dragging a progress bar) that require continuous, on-the-fly perception and adjustment. In this work, we develop ShowUI-$π$, the first flow-based generative model as GUI dexterous hand, featuring the following designs: (i) Unified Discrete-Continuous Actions, integrating discrete clicks and continuous drags within a shared model, enabling flexible adaptation across diverse interaction modes; (ii) Flow-based Action Generation for drag modeling, which predicts incremental cursor adjustments from continuous visual observations via a lightweight action expert, ensuring smooth and stable trajectories; (iii) Drag Training data and Benchmark, where we manually collect and synthesize 20K drag trajectories across five domains (e.g. PowerPoint, Adobe Premiere Pro), and introduce ScreenDrag, a benchmark with comprehensive online and offline evaluation protocols for assessing GUI agents' drag capabilities. Our experiments show that proprietary GUI agents still struggle on ScreenDrag (e.g. Operator scores 13.27, and the best Gemini-2.5-CUA reaches 22.18). In contrast, ShowUI-$π$ achieves 26.98 with only 450M parameters, underscoring both the difficulty of the task and the effectiveness of our approach. We hope this work advances GUI agents toward human-like dexterous control in digital world. The code is available at https://github.com/showlab/showui-pi.
Learning Dexterous Grasping from Sparse Taxonomy Guidance
Korea University(高丽大学);Korea University of Technology and Education(韩国技术教育大学);Naver AI Lab(Naver AI 实验室);Seoul National University(首尔大学);WIRobotics;RLWRLD
FunGrasp: Functional Grasping for Diverse Dexterous Hands
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));ETH Zürich(苏黎世联邦理工学院);The Hong Kong University of Science and Technology(香港科技大学)
ManiDext: Hand-Object Manipulation Synthesis via Continuous Correspondence Embeddings and Residual-Guided Diffusion
School of Electronic Engineering, Beijing University of Posts and Telecommunications(北京邮电大学电子工程学院);Department of Automation, Tsinghua University(清华大学自动化系);School of Artificial Intelligence, Beijing Normal University(北京师范大学人工智能学院)