Dexterous hands exhibit significant potential for complex real-world grasping tasks. While recent studies have primarily focused on learning policies for specific robotic hands, the development of a universal policy that controls diverse dexterous hands remains largely unexplored. In this work, we study the learning of cross-embodiment dexterous grasping policies using reinforcement learning (RL). Inspired by the capability of human hands to control various dexterous hands through teleoperation, we propose a universal action space based on the human hand's eigengrasps. The policy outputs eigengrasp actions that are then converted into specific joint actions for each robot hand through a retargeting mapping. We simplify the robot hand's proprioception to include only the positions of fingertips and the palm, offering a unified observation space across different robot hands. Our approach demonstrates an 80% success rate in grasping objects from the YCB dataset across four distinct embodiments using a single vision-based policy. Additionally, our policy exhibits zero-shot generalization to two previously unseen embodiments and significant improvement in efficient finetuning. For further details and videos, visit our project page https://sites.google.com/view/crossdex.
Reinforcement learning is a promising method for robotic grasping as it can learn effective reaching and grasping policies in difficult scenarios. However, achieving human-like manipulation capabilities with sophisticated robotic hands is challenging because of the problem's high dimensionality. Although remedies such as reward shaping or expert demonstrations can be employed to overcome this issue, they often lead to oversimplified and biased policies. We present Dext-Gen, a reinforcement learning framework for Dexterous Grasping in sparse reward ENvironments that is applicable to a variety o…
EaDex: A Cross-Embodiment Dexterous Manipulation Framework from Low-Cost Demonstrations
Faculty of Robot Science and Engineering, Northeastern University（机器人科学与工程学院，东北大学）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究所，中国科学院）；School of Automation, Nanjing University of Information Science and Technology（自动化学院，南京信息科学技术大学）
FunGrasp: Functional Grasping for Diverse Dexterous Hands
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；ETH Zürich（苏黎世联邦理工学院）；The Hong Kong University of Science and Technology（香港科技大学）
CADGrasp: Learning Contact and Collision Aware General Dexterous Grasping in Cluttered Scenes
Center on Frontiers of Computing Studies, School of Computer Science, Peking University（前沿计算研究中心，计算机学院，北京大学）；National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University（多媒体信息处理国家级重点实验室，计算机学院，北京大学）
DexRepNet: Learning Dexterous Robotic Grasping Network with Geometric and Spatial Hand-Object Representations
College of Control Science and Engineering, Zhejiang University（浙江大学控制科学与工程学院）；Key Lab of CS&AUS of Zhejiang Province（浙江省计算机与应用系统安全重点实验室）；Department of Computer Science, National University of Singapore（新加坡国立大学计算机科学系）
Cross-Embodiment Robot Manipulation via a Unified Hand Action Space
Intelligent Robotics and Vision Lab, University of Texas at Dallas（德克萨斯大学达拉斯分校智能机器人与视觉实验室）；Intelligent Robotics and Interactive Systems Lab, Arizona State University（亚利桑那州立大学智能机器人与交互系统实验室）
PartDexTOG: Generating Dexterous Task-Oriented Grasping via Language-driven Part Analysis
College of Computer Science and Technology, National University of Defense Technology（计算机科学与技术学院，国防科技大学）；College of Intelligence Science and Technology, National University of Defense Technology（智能科学与技术学院，国防科技大学）
SiMDex: Mining Similar Egocentric Videos for Cross-Embodiment Dexterous Manipulation
The University of Tokyo（东京大学）；ByteDance Seed（字节跳动 Seed）；The University of Hong Kong（香港大学）；Shanghai Jiao Tong University（上海交通大学）；Tsinghua University（清华大学）
Multi-Keypoint Affordance Representation for Functional Dexterous Grasping
School of Artificial Intelligence and Robotics, Hunan University（人工智能与机器人学院，湖南大学）；College of Computer Science and Electronic Engineering, Hunan University（计算机科学与电子工程学院，湖南大学）；National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University（机器人视觉感知与控制技术国家工程研究中心，湖南大学）
$\mathcal{D(R,O)}$ Grasp: A Unified Representation of Robot and Object Interaction for Cross-Embodiment Dexterous Grasping
National University of Singapore（新加坡国立大学）；School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University（上海交通大学电子信息与电气工程学院）；Zhiyuan College, Shanghai Jiao Tong University（上海交通大学致远学院）
MachaGrasp: Morphology-Aware Cross-Embodiment Dexterous Hand Articulation Generation for Grasping
Robotics & Autonomous Systems Division, Institute for Infocomm Research, Agency for Science, Technology and Research (A*STAR-I 2 R)（机器人与自主系统 division，信息通信研究所，科技研究局）；College of Computing and Data Science, Nanyang Technological University（计算与数据科学学院，南洋理工大学）；Show Lab, National University of Singapore（Show Lab，国立新加坡大学）
GRASPLAT: Enabling dexterous grasping through novel view synthesis
TeV, Fondazione Bruno Kessler（TeV，布鲁诺·凯瑟尔基金会）；PAVIS, Fondazione Istituto Italiano di Tecnologia（PAVIS，意大利技术研究院基金会）；ISR, Instituto Superior Técnico, Universidade de Lisboa（ISR，里斯本大学理工学院）；University of Trento（特伦托大学）
Istituto Italiano di Tecnologia（意大利技术研究院）；University of Naples Federico II（那不勒斯费德里科二世大学）
Manipulation in contrast to grasping is a trajectorial task that needs to use dexterous hands. Improving the dexterity of robot hands, increases the controller complexity and thus requires to use the concept of postural synergies. Inspired from postural synergies, this research proposes a new framework called kernelized synergies that focuses on the re-usability of the same subspace for precision grasping and dexterous manipulation. In this work, the computed subspace of postural synergies; parameterized by probabilistic movement primitives, is treated with kernel to preserve its grasping and …
