We present in-hand manipulation skills on a dexterous, compliant, anthropomorphic hand. Even though these skills were derived in a simplistic manner, they exhibit surprising robustness to variations in shape, size, weight, and placement of the manipulated object. They are also very insensitive to variation of execution speeds, ranging from highly dynamic to quasi-static. The robustness of the skills leads to compositional properties that enable extended and robust manipulation programs. To explain the surprising robustness of the in-hand manipulation skills, we performed a detailed, empirical analysis of the skills' performance. From this analysis, we identify three principles for skill design: 1) Exploiting the hardware's innate ability to drive hard-to-model contact dynamics. 2) Taking actions to constrain these interactions, funneling the system into a narrow set of possibilities. 3) Composing such action sequences into complex manipulation programs. We believe that these principles constitute an important foundation for robust robotic in-hand manipulation, and possibly for manipulation in general.
KTH Royal Institute of Technology（瑞典皇家理工学院）；Yale University（耶鲁大学）
This work focuses on the problem of in-hand manipulation and regrasping of objects with parallel grippers. We propose Dexterous Manipulation Graph (DMG) as a representation on which we define planning for in-hand manipulation and regrasping. The DMG is a disconnected undirected graph that represents the possible motions of a finger along the object's surface. We formulate the in-hand manipulation and regrasping problem as a graph search problem from the initial to the final configuration. The resulting plan is a sequence of coordinated in-hand pushing and regrasping movements. We propose a dua…
Osaka University（大阪大学）；Meiji University（明治大学）；National Institute of Advanced Industrial Science and Technology（产业技术综合研究所）
Synergy supplies a practical approach for expressing various postures of a multi-fingered hand. However, a conventional synergy defined for reproducing grasping postures cannot perform general-purpose tasks expected for a multi-fingered hand. Locking the position of particular fingers is essential for a multi-fingered hand to manipulate an object. When using conventional synergy based control to manipulate an object, which requires locking some fingers, the coordination of joints is heavily restricted, decreasing the dexterity of the hand. We propose a functionally divided manipulation synergy…
Dexterous Soft Hands Linearize Feedback-Control for In-Hand Manipulation
Robotics and Biology Laboratory, Technische Universität Berlin（柏林工业大学机器人与生物实验室）；Science of Intelligence, Research Cluster of Excellence, Berlin（柏林卓越研究集群智能科学）
KTH Royal Institute of Technology（KTH皇家理工学院）；University of Utah（犹他大学）；Yale University（耶鲁大学）；Google AI（谷歌人工智能）；NVIDIA Research（英伟达研究院）
The purpose of this benchmark is to evaluate the planning and control aspects of robotic in-hand manipulation systems. The goal is to assess the system's ability to change the pose of a hand-held object by either using the fingers, environment or a combination of both. Given an object surface mesh from the YCB data-set, we provide examples of initial and goal states (i.e.\ static object poses and fingertip locations) for various in-hand manipulation tasks. We further propose metrics that measure the error in reaching the goal state from a specific initial state, which, when aggregated across a…
Cross-Embodiment Robot Manipulation via a Unified Hand Action Space
Intelligent Robotics and Vision Lab, University of Texas at Dallas（德克萨斯大学达拉斯分校智能机器人与视觉实验室）；Intelligent Robotics and Interactive Systems Lab, Arizona State University（亚利桑那州立大学智能机器人与交互系统实验室）
Worcester Polytechnic Institute（伍斯特理工学院）；Imperial College London（伦敦帝国理工学院）
Attempts to achieve robotic Within-Hand-Manipulation (WIHM) generally utilize either high-DOF robotic hands with elaborate sensing apparatus or multi-arm robotic systems. In prior work we presented a simple robot hand with variable friction robot fingers, which allow a low-complexity approach to within-hand object translation and rotation, though this manipulation was limited to planar actions. In this work we extend the capabilities of this system to 3D manipulation with a novel region-based WIHM planning algorithm and utilizing extrinsic contacts. The ability to modulate finger friction enha…
Multi-fingered hands could be used to achieve many dexterous manipulation tasks, similarly to humans, and tactile sensing could enhance the manipulation stability for a variety of objects. However, tactile sensors on multi-fingered hands have a variety of sizes and shapes. Convolutional neural networks (CNN) can be useful for processing tactile information, but the information from multi-fingered hands needs an arbitrary pre-processing, as CNNs require a rectangularly shaped input, which may lead to unstable results. Therefore, how to process such complex shaped tactile information and utilize…
Institute of Robotics and Mechatronics, German Aerospace Center (DLR)（德国航空航天中心（DLR）机器人与机电一体化研究所）；Deggendorf Institute of Technology（代根多夫应用技术大学）
We show that a purely tactile dextrous in-hand manipulation task with continuous regrasping, requiring permanent force closure, can be learned from scratch and executed robustly on a torque-controlled humanoid robotic hand. The task is rotating a cube without dropping it, but in contrast to OpenAI's seminal cube manipulation task, the palm faces downwards and no cameras but only the hand's position and torque sensing are used. Although the task seems simple, it combines for the first time all the challenges in execution as well as learning that are important for using in-hand manipulation in r…
Progressive Transfer Learning for Dexterous In-Hand Manipulation with Multi-Fingered Anthropomorphic Hand
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Centre for Artificial Intelligence and Robotics, Hong Kong Institute of Science and Innovation, Chinese Academy of Sciences（中国科学院香港创新研究院人工智能与机器人中心）
TacGNN:Learning Tactile-based In-hand Manipulation with a Blind Robot
Southern University of Science and Technology（南方科技大学）；University of Hong Kong（香港大学）；Tencent Robotics X（腾讯 Robotics X）；University of Cambridge（剑桥大学）；Imperial College London（帝国理工学院）
We present the RBO Hand 3, a highly capable and versatile anthropomorphic soft hand based on pneumatic actuation. The RBO Hand 3 is designed to enable dexterous manipulation, to facilitate transfer of insights about human dexterity, and to serve as a robust research platform for extensive real-world experiments. It achieves these design goals by combining many degrees of actuation with intrinsic compliance, replicating relevant functioning of the human hand, and by combining robust components in a modular design. The RBO Hand 3 possesses 16 independent degrees of actuation, implemented in a de…
Massachusetts Institute of Technology（麻省理工学院）；Department of Mechanical Engineering（机械工程系（不提取，因为系属于细拆））
This paper presents an approach to in-hand manipulation planning that exploits the mechanics of alternating sticking contact. Particularly, we consider the problem of manipulating a grasped object using external pushes for which the pusher sticks to the object. Given the physical properties of the object, frictional coefficients at contacts and a desired regrasp on the object, we propose a sampling-based planning framework that builds a pushing strategy concatenating different feasible stable pushes to achieve the desired regrasp. An efficient dynamics formulation allows us to plan in-hand man…
Constraining contacts to remain fixed on an object during manipulation limits the potential workspace size, as motion is subject to the hand's kinematic topology. Finger gaiting is one way to alleviate such restraints. It allows contacts to be freely broken and remade so as to operate on different manipulation manifolds. This capability, however, has traditionally been difficult or impossible to practically realize. A finger gaiting system must simultaneously plan for and control forces on the object while maintaining stability during contact switching. This work alleviates the traditional req…
Do Robots Really Need Anthropomorphic Hands? A Comparison of Human and Robotic Hands
Robotics Innovation Center, Deutsches Forschungszentrum für Künstliche Intelligenz GmbH (DFKI)（德意志人工智能研究中心机器人创新中心）；Leibniz Universität Hannover, L3S Research Center（汉诺威莱布尼茨大学L3S研究中心）
Honda Research Institute USA（美国本田研究院）；Path Robotics Inc（Path机器人公司）；Stanford University（斯坦福大学）
This paper presents a hierarchical framework for planning and control of in-hand manipulation of a rigid object involving grasp changes using fully-actuated multifingered robotic hands. While the framework can be applied to the general dexterous manipulation, we focus on a more complex definition of in-hand manipulation, where at the goal pose the hand has to reach a grasp suitable for using the object as a tool. The high level planner determines the object trajectory as well as the grasp changes, i.e. adding, removing, or sliding fingers, to be executed by the low-level controller. While the …
In this paper, we present a method to manipulate unknown objects in-hand using tactile sensing without relying on a known object model. In many cases, vision-only approaches may not be feasible; for example, due to occlusion in cluttered spaces. We address this limitation by introducing a method to reorient unknown objects using tactile sensing. It incrementally builds a probabilistic estimate of the object shape and pose during task-driven manipulation. Our approach uses Bayesian optimization to balance exploration of the global object shape with efficient task completion. To demonstrate the …
Analyzing Key Objectives in Human-to-Robot Retargeting for Dexterous Manipulation
Department of Automation, Beijing National Research Center for Information Science and Technology, Tsinghua University, China（自动化系，信息科学与技术国家工程研究中心，清华大学）
We present a novel method for the direct transfer of grasps and manipulations between objects and hands through utilization of contact areas. Our method fully preserves contact shapes, and in contrast to existing techniques, is not dependent on grasp families, requires no model training or grasp sampling, makes no assumptions about manipulator morphology or kinematics, and allows user control over both transfer parameters and solution optimization. Despite these accommodations, we show that our method is capable of synthesizing kinematically feasible whole hand poses in seconds even for poor i…
