Designing robotic hands for specific tasks could help in the creation of optimized end-effectors for grasping and manipulation. However the systematic design of robotic hands for a simultaneous task of all fingertips presents many challenges. In this work the algorithms and implementation of an overall synthesis process is presented, which could be a first step towards a complete design tool for robotic end-effectors.
Type synthesis for a given task and number of fingers, solvability and dimensional synthesis for arbitrary topologies are developed and implemented. The resulting solver is a powerful tool that can aid in the creation of innovative robotic hands with arbitrary number of fingers and palms. Several examples of type synthesis, solvability calculations and dimensional synthesis are presented.
University of Edinburgh（爱丁堡大学）；University of Groningen（格罗宁根大学）；University College London（伦敦大学学院）
Generating high-quality instance-wise grasp configurations provides critical information of how to grasp specific objects in a multi-object environment and is of high importance for robot manipulation tasks. This work proposed a novel \textbf{S}ingle-\textbf{S}tage \textbf{G}rasp (SSG) synthesis network, which performs high-quality instance-wise grasp synthesis in a single stage: instance mask and grasp configurations are generated for each object simultaneously. Our method outperforms state-of-the-art on robotic grasp prediction based on the OCID-Grasp dataset, and performs competitively on t…
Dimensional synthesis refers to design of the dimensions of manipulators by optimising different kinds of performance indices. The motivation of this study is to perform dimensional synthesis for a wide set of spatial manipulators by optimising the manipulability of each manipulator around a pre-defined task point in the workspace and to finally give a prescription of manipulators along with their dimensions optimised for velocity and force transmission. A systematic method to formulate Jacobian matrix of a manipulator is presented. Optimisation of manipulability is performed for manipulation …
Automated chemical synthesis carries great promises of safety, efficiency and reproducibility for both research and industry laboratories. Current approaches are based on specifically-designed automation systems, which present two major drawbacks: (i) existing apparatus must be modified to be integrated into the automation systems; (ii) such systems are not flexible and would require substantial re-design to handle new reactions or procedures. In this paper, we propose a system based on a robot arm which, by mimicking the motions of human chemists, is able to perform complex chemical reactions…
We present a stepping stabilization control that addresses external push disturbances on bipedal walking robots. The stepping control is synthesized based on the step-to-step (S2S) dynamics of the robot that is controlled to have an approximately constant center of mass (COM) height. We first learn a linear S2S dynamics with bounded model discrepancy from the undisturbed walking behaviors of the robot, where the walking step size is taken as the control input to the S2S dynamics. External pushes are then considered as disturbances to the learned S2S (L-S2S) dynamics. We then apply the system-l…
AeroScene: Progressive Scene Synthesis for Aerial Robotics
University of Liverpool, UK（英国利物浦大学）；AIOZ Ltd., Singapore（新加坡AIOZ有限公司）；National Tsing Hua University, Taiwan（台湾国立清华大学）；RMIT University, Vietnam Campus（越南RMIT大学校园）；University of Information Technology, VNUHCM, Vietnam（越南VNUHCM大学信息科技学院）
Australian Centre for Robotic Vision（澳大利亚机器人视觉中心）；Queensland University of Technology（昆士兰科技大学）
This paper presents a real-time, object-independent grasp synthesis method which can be used for closed-loop grasping. Our proposed Generative Grasping Convolutional Neural Network (GG-CNN) predicts the quality and pose of grasps at every pixel. This one-to-one mapping from a depth image overcomes limitations of current deep-learning grasping techniques by avoiding discrete sampling of grasp candidates and long computation times. Additionally, our GG-CNN is orders of magnitude smaller while detecting stable grasps with equivalent performance to current state-of-the-art techniques. The light-we…
Instituto Balseiro（巴尔塞罗研究所）；Universidad Nacional de Cuyo（库约国立大学）
Robotic research over the last decades have lead us to different architectures to automatically synthesise discrete event controllers and implement these motion and task plans in real-world robot scenarios. However, these architectures usually build on existing robot hardware, generating as a result solutions that are influenced and/or restricted in their design by the available capabilities and sensors. In contrast to these approaches, we propose a design methodology that, given a specific domain of application, allowed us to build the first end-to-end implementation of an autonomous robot sy…
As robots gain capabilities to enter our human-centric world, they require formalism and algorithms that enable smart and efficient interactions. This is challenging, especially for robotic manipulators with complex tasks that may require collaboration with humans. Prior works approach this problem through reactive synthesis and generate strategies for the robot that guarantee task completion by assuming an adversarial human. While this assumption gives a sound solution, it leads to an "unfriendly" robot that is agnostic to the human intentions. We relax this assumption by formulating the prob…
In recent years, many works have investigated the feasibility of conversational robots for performing specific tasks, such as healthcare and interview. Along with this development comes a practical issue: how should we synthesize robotic voices to meet the needs of different situations? In this paper, we discuss this issue from three perspectives: 1) the difficulties of synthesizing non-verbal and interaction-oriented speech signals, particularly backchannels; 2) the scenario classification for robotic voice synthesis; 3) the ethical issues regarding the design of robot voice for its emotion a…
University of Waterloo（滑铁卢大学）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）
There has been increasing interest in smart factories powered by robotics systems to tackle repetitive, laborious tasks. One impactful yet challenging task in robotics-powered smart factory applications is robotic grasping: using robotic arms to grasp objects autonomously in different settings. Robotic grasping requires a variety of computer vision tasks such as object detection, segmentation, grasp prediction, pick planning, etc. While significant progress has been made in leveraging of machine learning for robotic grasping, particularly with deep learning, a big challenge remains in the need…
National Technical University of Athens（雅典国家技术大学）；Technische Universität Darmstadt（达姆施塔特工业大学）；Carnegie Mellon University（卡内基梅隆大学）
Inherent morphological characteristics in objects may offer a wide range of plausible grasping orientations that obfuscates the visual learning of robotic grasping. Existing grasp generation approaches are cursed to construct discontinuous grasp maps by aggregating annotations for drastically different orientations per grasping point. Moreover, current methods generate grasp candidates across a single direction in the robot's viewpoint, ignoring its feasibility constraints. In this paper, we propose a novel augmented grasp map representation, suitable for pixel-wise synthesis, that locally dis…
In this paper, we holistically present a Hybrid-Linear Inverted Pendulum (H-LIP) based approach for synthesizing and stabilizing 3D foot-underactuated bipedal walking, with an emphasis on thorough hardware realization. The H-LIP is proposed to capture the essential components of the underactuated and actuated part of the robotic walking. The robot walking gait is then directly synthesized based on the H-LIP. We comprehensively characterize the periodic orbits of the H-LIP and provably derive the stepping stabilization via its step-to-step (S2S) dynamics, which is then utilized to approximate t…
DenseReward: Dense Reward Learning via Failure Synthesis for Robotic Manipulation
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Carnegie Mellon University（卡内基梅隆大学）；Shanghai Jiao Tong University（上海交通大学）；Amazon AWS AI（亚马逊AWS人工智能）
RoboCodeX: Multimodal Code Generation for Robotic Behavior Synthesis
The University of Hong Kong（香港大学）；Shanghai AI Laboratory（上海人工智能实验室）；ETH Zurich（苏黎世联邦理工学院）；Shanghai Jiao Tong University（上海交通大学）；The Chinese University of Hong Kong（香港中文大学）；Tsinghua University（清华大学）
Osaka University（大阪大学）；H.U. Group Research Inst. G. K.（H.U.集团研究所有限责任公司）；National Inst. of AIST（日本产业技术综合研究所）
Deep learning methods have recently exhibited impressive performance in object detection. However, such methods needed much training data to achieve high recognition accuracy, which was time-consuming and required considerable manual work like labeling images. In this paper, we automatically prepare training data using robots. Considering the low efficiency and high energy consumption in robot motion, we proposed combining robotic in-hand observation and data synthesis to enlarge the limited data set collected by the robot. We first used a robot with a depth sensor to collect images of objects…
Deep Learning Approaches to Grasp Synthesis: A Review
Monash University（莫纳什大学）；The Australian National University（澳大利亚国立大学）；NVIDIA Corporation（英伟达公司）；LYRO Robotics Pty Ltd（LYRO机器人私人有限公司）；Stanford University（斯坦福大学）；Jaume I University（海梅一世大学）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；KTH Royal Institute of Technology（KTH皇家理工学院）；University of Washington（华盛顿大学）；Deakin University（迪肯大学）
ManiFoundation Model for General-Purpose Robotic Manipulation of Contact Synthesis with Arbitrary Objects and Robots
School of Electronics Engineering and Computer Science, Peking University（北京大学电子工程与计算机科学学院）；National University of Singapore（新加坡国立大学）；Shanghai Jiao Tong University（上海交通大学）
Expert demonstrations are a rich source of supervision for training visual robotic manipulation policies, but imitation learning methods often require either a large number of demonstrations or expensive online expert supervision to learn reactive closed-loop behaviors. In this work, we introduce SPARTN (Synthetic Perturbations for Augmenting Robot Trajectories via NeRF): a fully-offline data augmentation scheme for improving robot policies that use eye-in-hand cameras. Our approach leverages neural radiance fields (NeRFs) to synthetically inject corrective noise into visual demonstrations, us…
Peking University（北京大学）；Beijing Institute for General Artificial Intelligence（北京通用人工智能研究院）；Tsinghua University（清华大学）
Robotic dexterous grasping is the first step to enable human-like dexterous object manipulation and thus a crucial robotic technology. However, dexterous grasping is much more under-explored than object grasping with parallel grippers, partially due to the lack of a large-scale dataset. In this work, we present a large-scale robotic dexterous grasp dataset, DexGraspNet, generated by our proposed highly efficient synthesis method that can be generally applied to any dexterous hand. Our method leverages a deeply accelerated differentiable force closure estimator and thus can efficiently and robu…
University of Genoa（热那亚大学）；Italian Institute of Technology（意大利技术研究院）
Object manipulation is a natural activity we perform every day. How humans handle objects can communicate not only the willfulness of the acting, or key aspects of the context where we operate, but also the properties of the objects involved, without any need for explicit verbal description. Since human intelligence comprises the ability to read the context, allowing robots to perform actions that intuitively convey this kind of information would greatly facilitate collaboration. In this work, we focus on how to transfer on two different robotic platforms the same kinematics modulation that hu…
This document is an introduction to designing a multiple degree of freedom robotic manipulator. The goal of the robot is to sort an object based upon the objects color. The robot will be a synthesis of several linkages, servo motors, an Arduino system, and an end-effector. The design must be able to autonomously determine what object to move and where to place it. The robot will have a maximum reach of 24 horizontally. The operating conditions include being able to sort several different colored spheres within the area of a 180° up to 16 from the base of the arm. Sorting consists of picking up…
6D robotic grasping beyond top-down bin-picking scenarios is a challenging task. Previous solutions based on 6D grasp synthesis with robot motion planning usually operate in an open-loop setting, which are sensitive to grasp synthesis errors. In this work, we propose a new method for learning closed-loop control policies for 6D grasping. Our policy takes a segmented point cloud of an object from an egocentric camera as input, and outputs continuous 6D control actions of the robot gripper for grasping the object. We combine imitation learning and reinforcement learning and introduce a goal-auxi…
From Continuous Design to Delay-Aware Discrete Synthesis: Guaranteed High-Bandwidth Joint Control for PMSM Drives
Chair of Robotics and Systems Intelligence（机器人与系统智能主席职位）；Munich Institute of Robotics and Machine Intelligence (MIRMI)（慕尼黑机器人与机器智能研究所）；Technical University of Munich（慕尼黑工业大学）；Mohamed Bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
We propose a new algorithm to simplify the controller development for distributed robotic systems subject to external observations, disturbances, and communication delays. Unlike prior approaches that propose specialized solutions to handling communication latency for specific robotic applications, our algorithm uses an arbitrary centralized controller as the specification and automatically generates distributed controllers with communication management and delay compensation. We formulate our goal as nonlinear optimal control -- using a regret minimizing objective that measures how much the d…
University of Toronto（多伦多大学）；Vector Institute（向量研究所）；Nvidia（英伟达）；Samsung（三星）
The study of hand-object interaction requires generating viable grasp poses for high-dimensional multi-finger models, often relying on analytic grasp synthesis which tends to produce brittle and unnatural results. This paper presents Grasp'D, an approach for grasp synthesis with a differentiable contact simulation from both known models as well as visual inputs. We use gradient-based methods as an alternative to sampling-based grasp synthesis, which fails without simplifying assumptions, such as pre-specified contact locations and eigengrasps. Such assumptions limit grasp discovery and, in par…
InterMASH: A Unified Geometric Representation for Grasp Synthesis
University of Science and Technology of China（中国科学技术大学）；Jiangsu Key Laboratory of AI for Industries and Institute of AI for Industries Chinese Academy of Sciences（江苏省人工智能产业重点实验室及中国科学院人工智能产业研究院）；Shenzhen University（深圳大学）
Birla Institute of Technology and Science (BITS)（比尔拉理工学院）；Georgia Institute of Technology（佐治亚理工学院）
Control barrier functions are mathematical constructs used to guarantee safety for robotic systems. When integrated as constraints in a quadratic programming optimization problem, instantaneous control synthesis with real-time performance demands can be achieved for robotics applications. Prevailing use has assumed full knowledge of the safety barrier functions, however there are cases where the safe regions must be estimated online from sensor measurements. In these cases, the corresponding barrier function must be synthesized online. This paper describes a learning framework for estimating c…
Queensland University of Technology（昆士兰科技大学）；Australian Centre for Robotic Vision（澳大利亚机器人视觉中心）
We present the grasping system and design approach behind Cartman, the winning entrant in the 2017 Amazon Robotics Challenge. We investigate the design processes leading up to the final iteration of the system and describe the emergent solution by comparing it with key robotics design aspects. Following our experience, we propose a new design aspect, precision vs. redundancy, that should be considered alongside the previously proposed design aspects of modularity vs. integration, generality vs. assumptions, computation vs. embodiment and planning vs. feedback. We present the grasping system be…
