We introduce a large-scale dataset named MultiGripperGrasp for robotic grasping. Our dataset contains 30.4M grasps from 11 grippers for 345 objects. These grippers range from two-finger grippers to five-finger grippers, including a human hand. All grasps in the dataset are verified in the robot simulator Isaac Sim to classify them as successful and unsuccessful grasps. Additionally, the object fall-off time for each grasp is recorded as a grasp quality measurement. Furthermore, the grippers in our dataset are aligned according to the orientation and position of their palms, allowing us to transfer grasps from one gripper to another. The grasp transfer significantly increases the number of successful grasps for each gripper in the dataset. Our dataset is useful to study generalized grasp planning and grasp transfer across different grippers. Data, code and videos for the project are available at https://irvlutd.github.io/MultiGripperGrasp
Fraunhofer Institute for Manufacturing Engineering and Automation IPA（弗劳恩霍夫制造工程与自动化研究所IPA）；University of Stuttgart（斯图加特大学）
This paper presents a novel approach for the automatic offline grasp pose synthesis on known rigid objects for parallel jaw grippers. We use several criteria such as gripper stroke, surface friction, and a collision check to determine suitable 6D grasp poses on an object. In contrast to most available approaches, we neither aim for the best grasp pose nor for as many grasp poses as possible, but for a highly diverse set of grasps distributed all along the object. In order to accomplish this objective, we employ a clustering algorithm to the sampled set of grasps. This allows to simultaneously …
This paper presents a double jaw hand for industrial assembly. The hand comprises two orthogonal parallel grippers with different mechanisms. The inner gripper is made of a crank-slider mechanism which is compact and able to firmly hold objects like shafts. The outer gripper is made of a parallelogram that has large stroke to hold big objects like pulleys. The two grippers are connected by a prismatic joint along the hand's approaching vector. The hand is able to hold two objects and perform in-hand manipulation like pull-in (insertion) and push-out (ejection). This paper presents the detailed…
Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；The Chinese University of Hong Kong（香港中文大学）
In industry assembly lines, parts feeding machines are widely employed as the prologue of the whole procedure. They play the role of sorting the parts randomly placed in bins to the state with specified pose. With the help of the parts feeding machines, the subsequent assembly processes by robot arm can always start from the same condition. Thus it is expected that function of parting feeding machine and the robotic assembly can be integrated with one robot arm. This scheme can provide great flexibility and can also contribute to reduce the cost. The difficulties involved in this scheme lie in…
Technical University of Munich（慕尼黑工业大学）；Massachusetts Institute of Technology（麻省理工学院）
Robot grasping with deformable gripper jaws results in nonplanar surface contacts if the jaws deform to the nonplanar local geometry of an object. The frictional force and torque that can be transmitted through a nonplanar surface contact are both three-dimensional, resulting in a six-dimensional frictional wrench (6DFW). Applying traditional planar contact models to such contacts leads to over-conservative results as the models do not consider the nonplanar surface geometry and only compute a three-dimensional subset of the 6DFW. To address this issue, we derive the 6DFW for nonplanar surface…
The AUTOLAB at UC Berkeley（加州大学伯克利分校自动驾驶实验室）；Technical University of Munich（慕尼黑工业大学）
Robot grasping of deformable hollow objects such as plastic bottles and cups is challenging as the grasp should resist disturbances while minimally deforming the object so as not to damage it or dislodge liquids. We propose minimal work as a novel grasp quality metric that combines wrench resistance and the object deformation. We introduce an efficient algorithm to compute required work to resist an external wrench for a manipulation task by solving a linear program. The algorithm first computes the minimum required grasp force and an estimation of the gripper jaw displacements based on the ob…
This paper presents a sampling-based planning algorithm for in-hand manipulation of a grasped object using a series of external pushes. A high-level sampling-based planning framework, in tandem with a low-level inverse contact dynamics solver, effectively explores the space of continuous pushes with discrete pusher contact switch-overs. We model the frictional interaction between gripper, grasped object, and pusher, by discretizing complex surface/line contacts into arrays of hard frictional point contacts. The inverse dynamics problem of finding an instantaneous pusher motion that yields a de…
YUBI: Yielding Universal Bidigital Interface for Bimanual Dexterous Manipulation at Scale
AI Robot Association (AIRoA)（AI机器人协会）；Toyota Motor Corporation（丰田汽车公司）；National Institute of Advanced Industrial Science and Technology (AIST)（国家先进工业科学与技术研究院）；The University of Tokyo（东京大学）；Institute of Science Tokyo（东京科学研究所）；Waseda University（早稻田大学）
This paper presents a spectral correlation-based method (SpectGRASP) for robotic grasping of arbitrarily shaped, unknown objects. Given a point cloud of an object, SpectGRASP extracts contact points on the object's surface matching the hand configuration. It neither requires offline training nor a-priori object models. We propose a novel Binary Extended Gaussian Image (BEGI), which represents the point cloud surface normals of both object and robot fingers as signals on a 2-sphere. Spherical harmonics are then used to estimate the correlation between fingers and object BEGIs. The resulting spe…
University of Liverpool（利物浦大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Indian Institute of Science（印度科学学院）；The University of Tokyo（东京大学）；Zürcher Hochschule für Angewandte Wissenschaften（苏黎世应用科技大学）；University of Arkansas（阿肯色大学）；Physical Intelligence（物理智能公司）；Huazhong University of Science and Technology（华中科技大学）
Accurately simulating whether an object will be lifted securely or dropped during grasping is a longstanding Sim2Real challenge. Soft compliant jaw tips are almost universally used with parallel-jaw robot grippers due to their ability to increase contact area and friction between the jaws and the object to be manipulated. However, interactions between the compliant surfaces and rigid objects are notoriously difficult to model. We introduce IPC-GraspSim, a novel grasp simulator that extends Incremental Potential Contact (IPC) - a highly accurate collision + deformation model developed in 2020 f…
Designing Pin-pression Gripper and Learning its Dexterous Grasping with Online In-hand Adjustment
Dalian University of Technology（大连理工大学）；Wuhan University（武汉大学）；Shenyang University of Technology（沈阳理工大学）；National University of Defense Technology（国防科技大学）
We present pneumatic shape-shifting fingers to enable a simple parallel-jaw gripper for different manipulation modalities. By changing the finger geometry, the gripper effectively changes the contact type between the fingers and an object to facilitate distinct manipulation primitives. In this paper, we demonstrate the development and application of shape-shifting fingers to reorient and grasp cylindrical objects. The shape of the fingers changes based on the air pressure inside them and attains two distinct geometric forms at high and low pressure values. In our implementation, the finger sha…
Pickalo: Leveraging 6D Pose Estimation for Low-Cost Industrial Bin Picking
MESH Facility (formerly iCub Tech), Istituto Italiano di Tecnologia（意大利技术研究院MESH设施（原iCub Tech））；AMI laboratory, Istituto Italiano di Tecnologia（意大利技术研究院AMI实验室）；Institut des Systèmes Intelligents et de Robotique (ISIR)（智能系统与机器人研究所（ISIR））；Generative Bionics（Generative Bionics公司）；Camozzi Automation SpA（Camozzi自动化股份公司）
The 3D shape of a robot's end-effector plays a critical role in determining it's functionality and overall performance. Many industrial applications rely on task-specific gripper designs to ensure the system's robustness and accuracy. However, the process of manual hardware design is both costly and time-consuming, and the quality of the resulting design is dependent on the engineer's experience and domain expertise, which can easily be out-dated or inaccurate. The goal of this work is to use machine learning algorithms to automate the design of task-specific gripper fingers. We propose Fit2Fo…
Dyson Robotics Lab, Imperial College London（帝国理工学院戴森机器人实验室）
This paper presents a new method for parallel-jaw grasping of isolated objects from depth images, under large gripper pose uncertainty. Whilst most approaches aim to predict the single best grasp pose from an image, our method first predicts a score for every possible grasp pose, which we denote the grasp function. With this, it is possible to achieve grasping robust to the gripper's pose uncertainty, by smoothing the grasp function with the pose uncertainty function. Therefore, if the single best pose is adjacent to a region of poor grasp quality, that pose will no longer be chosen, and inste…
DeGrip: A Compact Cable-driven Robotic Gripper for Desktop Disassembly
J. Mike Walker ’66 Department of Mechanical Engineering, Texas A&M University（J. Mike Walker ’66 机械工程系，德克萨斯A&M大学）；Zachry Department of Civil and Environmental Engineering, Texas A&M University（扎克里 civil 和环境工程系，德克萨斯A&M大学）
本文提出紧凑型绳索驱动三自由度夹爪 DeGrip，通过解耦腕部与夹爪驱动支持狭窄空间内的任意构型台式机拆解，并在 Isaac Sim 环境中验证其有效性。
Soft optical tactile sensors enable robots to manipulate deformable objects by capturing important features such as high-resolution contact geometry and estimations of object compliance. This work presents a variable stiffness soft tactile end-effector called StRETcH, a Soft to Resistive Elastic Tactile Hand, that is easily manufactured and integrated with a robotic arm. An elastic membrane is suspended between two robotic fingers, and a depth sensor capturing the deformations of the elastic membrane enables sub-millimeter accurate estimates of contact geometries. The parallel-jaw gripper vari…
Vacuum-based end effectors are widely used in industry and are often preferred over parallel-jaw and multifinger grippers due to their ability to lift objects with a single point of contact. Suction grasp planners often target planar surfaces on point clouds near the estimated centroid of an object. In this paper, we propose a compliant suction contact model that computes the quality of the seal between the suction cup and local target surface and a measure of the ability of the suction grasp to resist an external gravity wrench. To characterize grasps, we estimate robustness to perturbations …
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；University of Waterloo（滑铁卢大学）
Autonomous bin picking poses significant challenges to vision-driven robotic systems given the complexity of the problem, ranging from various sensor modalities, to highly entangled object layouts, to diverse item properties and gripper types. Existing methods often address the problem from one perspective. Diverse items and complex bin scenes require diverse picking strategies together with advanced reasoning. As such, to build robust and effective machine-learning algorithms for solving this complex task requires significant amounts of comprehensive and high quality data. Collecting such dat…
We study the problem of imitating object interactions from Internet videos. This requires understanding the hand-object interactions in 4D, spatially in 3D and over time, which is challenging due to mutual hand-object occlusions. In this paper we make two main contributions: (1) a novel reconstruction technique RHOV (Reconstructing Hands and Objects from Videos), which reconstructs 4D trajectories of both the hand and the object using 2D image cues and temporal smoothness constraints; (2) a system for imitating object interactions in a physics simulator with reinforcement learning. We apply ou…
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences（中国科学院脑科学与智能技术卓越创新中心）；Centre for Artificial Intelligence and Robotics, Hong Kong Institute of Science and Innovation, Chinese Academy of Sciences（中国科学院香港创新研究院人工智能与机器人中心）
Grasping with anthropomorphic robotic hands involves much more hand-object interactions compared to parallel-jaw grippers. Modeling hand-object interactions is essential to the study of multi-finger hand dextrous manipulation. This work presents DVGG, an efficient grasp generation network that takes single-view observation as input and predicts high-quality grasp configurations for unknown objects. In general, our generative model consists of three components: 1) Point cloud completion for the target object based on the partial observation; 2) Diverse sets of grasps generation given the comple…
University of Applied Sciences Augsburg（奥格斯堡应用技术大学）；Ulster University（阿尔斯特大学）
A robust grip is key to successful manipulation and joining of work pieces involved in any industrial assembly process. Stability of a grip depends on geometric and physical properties of the object as well as the gripper itself. Current state-of-the-art algorithms can usually predict if a grip would fail. However, they are not able to predict the force at which the gripped object starts to slip, which is critical as the object might be subjected to external forces, e.g. when joining it with another object. This research project aims to develop a AI-based approach for a grip metric based on ta…
University of Michigan（密歇根大学）；Google（谷歌）；Google Brain（谷歌大脑）；X Inc（X公司）
This paper focuses on the problem of learning 6-DOF grasping with a parallel jaw gripper in simulation. We propose the notion of a geometry-aware representation in grasping based on the assumption that knowledge of 3D geometry is at the heart of interaction. Our key idea is constraining and regularizing grasping interaction learning through 3D geometry prediction. Specifically, we formulate the learning of deep geometry-aware grasping model in two steps: First, we learn to build mental geometry-aware representation by reconstructing the scene (i.e., 3D occupancy grid) from RGBD input via gener…
