Humans can determine a proper strategy to grasp an object according to the measured physical attributes or the prior knowledge of the object. This paper proposes an approach to determining the strategy of dexterous grasping by using an anthropomorphic robotic hand simply based on a label or a description of an object. Object attributes are parsed from natural-language descriptions and augmented with an object knowledge base that is scraped from retailer websites. A novel metric named joint probability distance is defined to measure distance between object attributes. The probability distribution of grasp types for the given object is learned using a deep neural network which takes in object features as input. The action of the multi-fingered hand with redundant degrees of freedom (DoF) is controlled by a linear inverse-kinematics model of grasp topology and scales. The grasping strategy generated by the proposed approach is evaluated both by simulation and execution on a Sawyer robot with an AR10 robotic hand.
DexTele: A Dual-Arm Dexterous Teleoperation System Based on Motion Retargeting and Adaptive Force Control
School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen（人工智能学院，香港中文大学（深圳））；School of Computing, University of Portsmouth（计算学院，朴茨茅斯大学）
T(R,O) Grasp: Efficient Graph Diffusion of Robot-Object Spatial Transformation for Cross-Embodiment Dexterous Grasping
Department of Computer Science, National University of Singapore（新加坡国立大学计算机科学系）；RoboScience；College of Control Science and Engineering, Zhejiang University（浙江大学控制科学与工程学院）
Peking University（北京大学）；Beijing Institute for General AI（北京通用人工智能研究院）；Stanford University（斯坦福大学）；Tsinghua University（清华大学）
In this work, we tackle the problem of learning universal robotic dexterous grasping from a point cloud observation under a table-top setting. The goal is to grasp and lift up objects in high-quality and diverse ways and generalize across hundreds of categories and even the unseen. Inspired by successful pipelines used in parallel gripper grasping, we split the task into two stages: 1) grasp proposal (pose) generation and 2) goal-conditioned grasp execution. For the first stage, we propose a novel probabilistic model of grasp pose conditioned on the point cloud observation that factorizes rota…
How should a robot direct active vision so as to ensure reliable grasping? We answer this question for the case of dexterous grasping of unfamiliar objects. By dexterous grasping we simply mean grasping by any hand with more than two fingers, such that the robot has some choice about where to place each finger. Such grasps typically fail in one of two ways, either unmodeled objects in the scene cause collisions or object reconstruction is insufficient to ensure that the grasp points provide a stable force closure. These problems can be solved more easily if active sensing is guided by the anti…
Reinforcement learning is a promising method for robotic grasping as it can learn effective reaching and grasping policies in difficult scenarios. However, achieving human-like manipulation capabilities with sophisticated robotic hands is challenging because of the problem's high dimensionality. Although remedies such as reward shaping or expert demonstrations can be employed to overcome this issue, they often lead to oversimplified and biased policies. We present Dext-Gen, a reinforcement learning framework for Dexterous Grasping in sparse reward ENvironments that is applicable to a variety o…
CADGrasp: Learning Contact and Collision Aware General Dexterous Grasping in Cluttered Scenes
Center on Frontiers of Computing Studies, School of Computer Science, Peking University（前沿计算研究中心，计算机学院，北京大学）；National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University（多媒体信息处理国家级重点实验室，计算机学院，北京大学）
DexRepNet: Learning Dexterous Robotic Grasping Network with Geometric and Spatial Hand-Object Representations
College of Control Science and Engineering, Zhejiang University（浙江大学控制科学与工程学院）；Key Lab of CS&AUS of Zhejiang Province（浙江省计算机与应用系统安全重点实验室）；Department of Computer Science, National University of Singapore（新加坡国立大学计算机科学系）
Dexterous robotic hands have the capability to interact with a wide variety of household objects to perform tasks like grasping. However, learning robust real world grasping policies for arbitrary objects has proven challenging due to the difficulty of generating high quality training data. In this work, we propose a learning system (ISAGrasp) for leveraging a small number of human demonstrations to bootstrap the generation of a much larger dataset containing successful grasps on a variety of novel objects. Our key insight is to use a correspondence-aware implicit generative model to deform ob…
PartDexTOG: Generating Dexterous Task-Oriented Grasping via Language-driven Part Analysis
College of Computer Science and Technology, National University of Defense Technology（计算机科学与技术学院，国防科技大学）；College of Intelligence Science and Technology, National University of Defense Technology（智能科学与技术学院，国防科技大学）
UltraDexGrasp: Learning Universal Dexterous Grasping for Bimanual Robots with Synthetic Data
Shanghai AI Laboratory（上海人工智能实验室）；The Chinese University of Hong Kong（香港中文大学）；Zhejiang University（浙江大学）；The University of Hong Kong（香港大学）；Peking University（北京大学）
Variational Neural Belief Parameterizations for Robust Dexterous Grasping under Multimodal Uncertainty
Department of Electrical & Computer Engineering and Institute for Systems Research, University of Maryland, College Park, MD, USA（电气与计算机工程系和系统研究所，马里兰大学，College Park, MD, USA）；Maryland Applied Graduate Engineering, A. James Clark School of Engineering, University of Maryland, College Park, MD, USA（马里兰应用研究生工程学院，A. James Clark工程学院，马里兰大学，College Park, MD, USA）
ClutterDexGrasp: A Sim-to-Real System for General Dexterous Grasping in Cluttered Scenes
CFCS, School of Computer Science, Peking University（计算机学院，北京大学）；PKU-AgiBot Lab（北京大学阿吉机器人实验室）；PKU-PsiBot Lab（北京大学Psi机器人实验室）；Princeton University（普林斯顿大学）
FunGrasp: Functional Grasping for Diverse Dexterous Hands
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；ETH Zürich（苏黎世联邦理工学院）；The Hong Kong University of Science and Technology（香港科技大学）
$\mathcal{D(R,O)}$ Grasp: A Unified Representation of Robot and Object Interaction for Cross-Embodiment Dexterous Grasping
National University of Singapore（新加坡国立大学）；School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University（上海交通大学电子信息与电气工程学院）；Zhiyuan College, Shanghai Jiao Tong University（上海交通大学致远学院）
We propose an approach to multi-modal grasp detection that jointly predicts the probabilities that several types of grasps succeed at a given grasp pose. Given a partial point cloud of a scene, the algorithm proposes a set of feasible grasp candidates, then estimates the probabilities that a grasp of each type would succeed at each candidate pose. Predicting grasp success probabilities directly from point clouds makes our approach agnostic to the number and placement of depth sensors at execution time. We evaluate our system both in simulation and on a real robot with a Robotiq 3-Finger Adapti…
Grasp What You Want: Embodied Dexterous Grasping System Driven by Your Voice
Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳市人工智能与机器人研究院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；South China University of Technology（华南理工大学）
MoDex: A Diffusion Policy for Sequential Multi-Object Dexterous Grasping
Department of Robotics, Perception and Learning, KTH Royal Institute of Technology（机器人、感知与学习系，皇家理工学院）；Robotics and Autonomous Systems at University of Turku（图尔库大学机器人与自主系统）
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
While motion planning of locomotion for legged robots has shown great success, motion planning for legged robots with dexterous multi-finger grasping is not mature yet. We present an efficient motion planning framework for simultaneously solving locomotion (e.g., centroidal dynamics), grasping (e.g., patch contact), and contact (e.g., gait) problems. To accelerate the planning process, we propose distributed optimization frameworks based on Alternating Direction Methods of Multipliers (ADMM) to solve the original large-scale Mixed-Integer NonLinear Programming (MINLP). The resulting frameworks…
Beijing Institute of General Artificial Intelligence (BIGAI)（北京通用人工智能研究院）；Tsinghua University（清华大学）；Peking University（北京大学）
Generating dexterous grasping has been a long-standing and challenging robotic task. Despite recent progress, existing methods primarily suffer from two issues. First, most prior arts focus on a specific type of robot hand, lacking the generalizable capability of handling unseen ones. Second, prior arts oftentimes fail to rapidly generate diverse grasps with a high success rate. To jointly tackle these challenges with a unified solution, we propose GenDexGrasp, a novel hand-agnostic grasping algorithm for generalizable grasping. GenDexGrasp is trained on our proposed large-scale multi-hand gra…
Designing Pin-pression Gripper and Learning its Dexterous Grasping with Online In-hand Adjustment
Dalian University of Technology（大连理工大学）；Wuhan University（武汉大学）；Shenyang University of Technology（沈阳理工大学）；National University of Defense Technology（国防科技大学）
