Deep Reinforcement Learning (DRL) is emerging as a promising approach to generate adaptive behaviors for robotic platforms. However, a major drawback of using DRL is the data-hungry training regime that requires millions of trial and error attempts, which is impractical when running experiments on robotic systems. Learning from Demonstrations (LfD) has been introduced to solve this issue by cloning the behavior of expert demonstrations. However, LfD requires a large number of demonstrations that are difficult to be acquired since dedicated complex setups are required. To overcome these limitations, we propose a multi-subtask reinforcement learning methodology where complex pick and place tasks can be decomposed into low-level subtasks. These subtasks are parametrized as expert networks and learned via DRL methods. Trained subtasks are then combined by a high-level choreographer to accomplish the intended pick and place task considering different initial configurations. As a testbed, we use a pick and place robotic simulator to demonstrate our methodology and show that our method outperforms a benchmark methodology based on LfD in terms of sample-efficiency. We transfer the learned policy to the real robotic system and demonstrate robust grasping using various geometric-shaped objects.
Flexible pick-and-place is a fundamental yet challenging task within robotics, in particular due to the need of an object model for a simple target pose definition. In this work, the robot instead learns to pick-and-place objects using planar manipulation according to a single, demonstrated goal state. Our primary contribution lies within combining robot learning of primitives, commonly estimated by fully-convolutional neural networks, with one-shot imitation learning. Therefore, we define the place reward as a contrastive loss between real-world measurements and a task-specific noise distribu…
Pick-and-place Manipulation Across Grippers Without Retraining: A Learning-optimization Diffusion Policy Approach
Technical University of Munich（慕尼黑工业大学）；Hong Kong University of Science and Technology（香港科技大学）；Nanjing University（南京大学）；Sun Yat-sen University（中山大学）；Tsinghua University（清华大学）
Complex object manipulation tasks often span over long sequences of operations. Task planning over long-time horizons is a challenging and open problem in robotics, and its complexity grows exponentially with an increasing number of subtasks. In this paper we present a deep learning network that learns dependencies and transitions across subtasks solely from a set of demonstration videos. We represent each subtask as an action symbol (e.g. move cup), and show that these symbols can be learned and predicted directly from image observations. Learning from demonstrations and visual observations a…
Georgia Institute of Technology（佐治亚理工学院）；Meta AI Research（Meta人工智能研究院）
We present a large-scale study of imitating human demonstrations on tasks that require a virtual robot to search for objects in new environments -- (1) ObjectGoal Navigation (e.g. 'find & go to a chair') and (2) Pick&Place (e.g. 'find mug, pick mug, find counter, place mug on counter'). First, we develop a virtual teleoperation data-collection infrastructure -- connecting Habitat simulator running in a web browser to Amazon Mechanical Turk, allowing remote users to teleoperate virtual robots, safely and at scale. We collect 80k demonstrations for ObjectNav and 12k demonstrations for Pick&Place…
Karlsruhe University of Applied Sciences（卡尔斯鲁厄应用技术大学）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）
Precise pick-and-place is essential in robotic applications. To this end, we define a novel exact training method and an iterative inference method that improve pick-and-place precision with Transporter Networks. We conduct a large scale experiment on 8 simulated tasks. A systematic analysis shows, that the proposed modifications have a significant positive effect on model performance. Considering picking and placing independently, our methods achieve up to 60% lower rotation and translation errors than baselines. For the whole pick-and-place process we observe 50% lower rotation errors for mo…
Robots have the capability to collect large amounts of data autonomously by interacting with objects in the world. However, it is often not obvious \emph{how} to learning from autonomously collected data without human-labeled supervision. In this work we learn pixel-wise object representations from unsupervised pick and place data that generalize to new objects. We introduce a novel framework for using these representations in order to predict where to pick and where to place in order to match a goal image. Finally, we demonstrate the utility of our approach in a simulated grasping environment…
Northeastern University（东北大学）；University of Massachusetts Lowell（马萨诸塞大学洛厄尔分校）
Assistive robot manipulators must be able to autonomously pick and place a wide range of novel objects to be truly useful. However, current assistive robots lack this capability. Additionally, assistive systems need to have an interface that is easy to learn, to use, and to understand. This paper takes a step forward in this direction. We present a robot system comprised of a robotic arm and a mobility scooter that provides both pick-and-drop and pick-and-place functionality for open world environments without modeling the objects or environment. The system uses a laser pointer to directly sel…
College of Computer and Information Science, Northeastern University（东北大学计算机与信息科学学院）
We propose a novel formulation of robotic pick and place as a deep reinforcement learning (RL) problem. Whereas most deep RL approaches to robotic manipulation frame the problem in terms of low level states and actions, we propose a more abstract formulation. In this formulation, actions are target reach poses for the hand and states are a history of such reaches. We show this approach can solve a challenging class of pick-place and regrasping problems where the exact geometry of the objects to be handled is unknown. The only information our method requires is: 1) the sensor perception availab…
In this paper, we investigate the possibility of applying plan transformations to general manipulation plans in order to specialize them to the specific situation at hand. We present a framework for optimizing execution and achieving higher performance by autonomously transforming robot's behavior at runtime. We show that plans employed by robotic agents in real-world environments can be transformed, despite their control structures being very complex due to the specifics of acting in the real world. The evaluation is carried out on a plan of a PR2 robot performing pick and place tasks, to whi…
We present a behaviour-based reinforcement learning approach, inspired by Brook's subsumption architecture, in which simple fully connected networks are trained as reactive behaviours. Our working assumption is that a pick and place robotic task can be simplified by leveraging domain knowledge of a robotics developer to decompose and train such reactive behaviours; namely, approach, grasp, and retract. Then the robot autonomously learns how to combine them via an Actor-Critic architecture. The Actor-Critic policy is to determine the activation and inhibition mechanisms of the reactive behaviou…
The MineRL competition is designed for the development of reinforcement learning and imitation learning algorithms that can efficiently leverage human demonstrations to drastically reduce the number of environment interactions needed to solve the complex \emph{ObtainDiamond} task with sparse rewards. To address the challenge, in this paper, we present \textbf{SEIHAI}, a \textbf{S}ample-\textbf{e}ff\textbf{i}cient \textbf{H}ierarchical \textbf{AI}, that fully takes advantage of the human demonstrations and the task structure. Specifically, we split the task into several sequentially dependent s…
College of Computer and Information Science（计算机与信息科学学院）；Northeastern University（东北大学）
We address a class of manipulation problems where the robot perceives the scene with a depth sensor and can move its end effector in a space with six degrees of freedom -- 3D position and orientation. Our approach is to formulate the problem as a Markov decision process (MDP) with abstract yet generally applicable state and action representations. Finding a good solution to the MDP requires adding constraints on the allowed actions. We develop a specific set of constraints called hierarchical $\text{SE}(3)$ sampling (HSE3S) which causes the robot to learn a sequence of gazes to focus attention…
Singapore Centre for 3D Printing（新加坡3D打印中心）；Nanyang Technological University（南洋理工大学）；Eureka Robotics Pte. Ltd.（尤里卡机器人私人有限公司）
Fast robotics pick-and-place with suction cups is a crucial component in the current development of automation in logistics (factory lines, e-commerce, etc.). By "critically fast" we mean the fastest possible movement for transporting an object such that it does not slip or fall from the suction cup. The main difficulties are: (i) handling the contact between the suction cup and the object, which fundamentally involves kinodynamic constraints; and (ii) doing so at a low computational cost, typically a few hundreds of milliseconds. To address these difficulties, we propose (a) a model for sucti…
This document is an introduction to designing a multiple degree of freedom robotic manipulator. The goal of the robot is to sort an object based upon the objects color. The robot will be a synthesis of several linkages, servo motors, an Arduino system, and an end-effector. The design must be able to autonomously determine what object to move and where to place it. The robot will have a maximum reach of 24 horizontally. The operating conditions include being able to sort several different colored spheres within the area of a 180° up to 16 from the base of the arm. Sorting consists of picking up…
Rutgers, the State University of New Jersey（罗格斯新泽西州立大学）
Robotic pick-and-place (PnP) operations on moving conveyors find a wide range of industrial applications. In practice, simple greedy heuristics (e.g., prioritization based on the time to process a single object) are applied that achieve reasonable efficiency. We show analytically that, under a simplified telescoping robot model, these greedy approaches do not ensure time optimality of PnP operations. To address the shortcomings of classical solutions, we develop algorithms that compute optimal object picking sequences for a predetermined finite horizon. Employing dynamic programming techniques…
Khoury College of Computer Sciences, Northeastern University（东北大学科里计算机科学学院）
We consider robotic pick-and-place of partially visible, novel objects, where goal placements are non-trivial, e.g., tightly packed into a bin. One approach is (a) use object instance segmentation and shape completion to model the objects and (b) use a regrasp planner to decide grasps and places displacing the models to their goals. However, it is critical for the planner to account for uncertainty in the perceived models, as object geometries in unobserved areas are just guesses. We account for perceptual uncertainty by incorporating it into the regrasp planner's cost function. We compare sev…
Automation applications are pushing the deployment of many high DoF manipulators in warehouse and manufacturing environments. This has motivated many efforts on optimizing manipulation tasks involving a single arm. Coordinating multiple arms for manipulation, however, introduces additional computational challenges arising from the increased DoFs, as well as the combinatorial increase in the available operations that many manipulators can perform, including handoffs between arms. The focus here is on the case of pick-and-place tasks, which require a sequence of handoffs to be executed, so as to…
Recent progress in robotic manipulation has dealt with the case of previously unknown objects in the context of relatively simple tasks, such as bin-picking. Existing methods for more constrained problems, however, such as deliberate placement in a tight region, depend more critically on shape information to achieve safe execution. This work deals with pick-and-constrained placement of objects without access to geometric models. The objective is to pick an object and place it safely inside a desired goal region without any collisions, while minimizing the time and the sensing operations requir…
Soft robotic systems present a variety of new opportunities for solving complex problems. The use of soft robotic grippers, for example, can simplify the complexity in tasks such as the of grasping irregular and delicate objects. Adoption of soft robotics by academia and industry, however, has been slow and this is, in-part, due to the amount of hardware and software that must be developed from scratch for each use of soft system components. In this paper we detail the design, fabrication and validation of an open-source framework that we designed to lower the barrier to entry for integrating …
Australian Centre for Robotic Vision (ACRV)（澳大利亚机器人视觉中心（ACRV））；Queensland University of Technology (QUT)（昆士兰科技大学（QUT））；University of Adelaide（阿德莱德大学）；Australian National University (ANU)（澳大利亚国立大学（ANU））
Robotic manipulation and grasping in cluttered and unstructured environments is a current challenge for robotics. Enabling robots to operate in these challenging environments have direct applications from automating warehouses to harvesting fruit in agriculture. One of the main challenges associated with these difficult robotic manipulation tasks is the motion planning and control problem for multi-DoF (Degree of Freedom) manipulators. This paper presents the design and performance evaluation of a low-cost Cartesian manipulator, Cartman who took first place in the Amazon Robotics Challenge 201…
Programming robots is a complicated and time-consuming task. A robot is essentially a real-time, distributed embedded system. Often, control and communication paths within the system are tightly coupled to the actual physical configuration of the robot. Thus, programming a robot is a very challenging task for domain experts who do not have a dedicated background in robotics. In this paper we present an approach towards a domain specific language, which is intended to reduce the efforts and the complexity which is required when developing robotic applications. Furthermore we apply a software pr…
Osaka University（大阪大学）；National Institute of Advanced Industrial Science and Technology (AIST)（国立先进工业科学技术研究所（AIST））
In this paper, we present a planner that plans a sequence of base positions for a mobile manipulator to efficiently and robustly collect objects stored in distinct trays. We achieve high efficiency by exploring the common areas where a mobile manipulator can grasp objects stored in multiple trays simultaneously and move the mobile manipulator to the common areas to reduce the time needed for moving the mobile base. We ensure robustness by optimizing the base position with the best clearance to positioning uncertainty so that a mobile manipulator can complete the task even if there is a certain…
The University of Manchester（曼彻斯特大学）；Cardiff University（卡迪夫大学）
This paper presents the development of a control system for vision-guided pick-and-place tasks using a robot arm equipped with a 3D camera. The main steps include camera intrinsic and extrinsic calibration, hand-eye calibration, initial object pose registration, objects pose alignment algorithm, and pick-and-place execution. The proposed system allows the robot be able to to pick and place object with limited times of registering a new object and the developed software can be applied for new object scenario quickly. The integrated system was tested using the hardware combination of kuka iiwa, …
Dorabot Inc.（多拉机器人公司）；University of Hong Kong（香港大学）；Advanced Industrial Science and Technology（日本产业技术综合研究所）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；City University of Hong Kong（香港城市大学）
Robots that autonomously manipulate objects within warehouses have the potential to shorten the package delivery time and improve the efficiency of the e-commerce industry. In this paper, we present a robotic system that is capable of both picking and placing general objects in warehouse scenarios. Given a target object, the robot autonomously detects it from a shelf or a table and estimates its full 6D pose. With this pose information, the robot picks the object using its gripper, and then places it into a container or at a specified location. We describe our pick-and-place system in detail w…
In an environment where a manipulator needs to execute multiple consecutive tasks, the act of object manoeuvre will change the underlying configuration space, affecting all subsequent tasks. Previously free configurations might now be occupied by the manoeuvred objects, and previously occupied space might now open up new paths. We propose Lazy Tree-based Replanner (LTR*) -- a novel hybrid planner that inherits the rapid planning nature of existing anytime incremental sampling-based planners. At the same time, it allows subsequent tasks to leverage prior experience via a lazy experience graph. …
In this paper, we propose a novel edge and corner detection algorithm for an unorganized point cloud. Our edge detection method classifies a query point as an edge point by evaluating the distribution of local neighboring points around the query point. The proposed technique has been tested on generic items such as dragons, bunnies, and coffee cups from the Stanford 3D scanning repository. The proposed technique can be directly applied to real and unprocessed point cloud data of random clutter of objects. To demonstrate the proposed technique's efficacy, we compare it to the other solutions fo…
Central South University（中南大学）；Norwegian University of Science & Technology (NTNU)（挪威科技大学）
The use of robotics in food industry is becoming more popular in recent years. The trend seems to continue as long as the robotics technology meets diverse and challenging needs of the food producers. Rapid developments in digital computers and control systems technologies have significant impact in robotics like any other engineering fields. By utilizing new hardware and software tools, design of these complex systems that need strong integration of distinct disciplines is no longer difficult compared to the past. Therefore, the purpose of this paper is to design and implement a micro-control…
Data Science Research Laboratories, NEC Corporation（日本电气股份有限公司数据科学研究实验室）
We present new models of optimization-based task and motion planning (TAMP) for robotic pick-and-place (P&P), which plan action sequences and motion trajectory with low computational costs. We improved an existing state-of-the-art TAMP model integrated with the collision avoidance, which is formulated as a mixed-integer linear programing (MILP) problem. To enable the MILP solver to search for solutions efficiently, we introduced two approaches leveraging features of collision avoidance in robotic P&P. The first approach reduces number of binary variables, which are related to the collision avo…
We present joint learning of instance and semantic segmentation for visible and occluded region masks. Sharing the feature extractor with instance occlusion segmentation, we introduce semantic occlusion segmentation into the instance segmentation model. This joint learning fuses the instance- and image-level reasoning of the mask prediction on the different segmentation tasks, which was missing in the previous work of learning instance segmentation only (instance-only). In the experiments, we evaluated the proposed joint learning comparing the instance-only learning on the test dataset. We als…
Optimized Coordination Strategy for Multi-Aerospace Systems in Pick-and-Place Tasks By Deep Neural Network
University of Pittsburgh（匹兹堡大学）；University of California, Irvine（加州大学欧文分校）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；New York University（纽约大学）；George Washington University（乔治·华盛顿大学）
A Mosquito Pick-and-Place System for PfSPZ-based Malaria Vaccine Production
Johns Hopkins University（约翰斯·霍普金斯大学）；Georgia Tech（佐治亚理工学院）；Accenture（埃森哲）；Auris Health（奥里斯健康）；Technical University of Munich（慕尼黑工业大学）；National University of Singapore（新加坡国立大学）；Sanaria, Inc.（萨纳利亚公司）
University of Freiburg（弗赖堡大学）；Toyota Research Institute（丰田研究院）
Controlling robots to perform tasks via natural language is one of the most challenging topics in human-robot interaction. In this work, we present a robot system that follows unconstrained language instructions to pick and place arbitrary objects and effectively resolves ambiguities through dialogues. Our approach infers objects and their relationships from input images and language expressions and can place objects in accordance with the spatial relations expressed by the user. Unlike previous approaches, we consider grounding not only for the picking but also for the placement of everyday o…
Rutgers University（罗格斯大学）；Federal University of Espirito Santo（圣埃斯皮里图联邦大学）
An important logistics application of robotics involves manipulators that pick-and-place objects placed in warehouse shelves. A critical aspect of this task corre- sponds to detecting the pose of a known object in the shelf using visual data. Solving this problem can be assisted by the use of an RGB-D sensor, which also provides depth information beyond visual data. Nevertheless, it remains a challenging problem since multiple issues need to be addressed, such as low illumination inside shelves, clutter, texture-less and reflective objects as well as the limitations of depth sensors. This pape…
National Institute of Advanced Industrial Science and Technology (AIST)（国立先进工业科学技术研究所（AIST））；Osaka University（大阪大学）
In this paper, we address efficiently and robustly collecting objects stored in different trays using a mobile manipulator. A resolution complete method, based on precomputed reachability database, is proposed to explore collision-free inverse kinematics (IK) solutions and then a resolution complete set of feasible base positions can be determined. This method approximates a set of representative IK solutions that are especially helpful when solving IK and checking collision are treated separately. For real world applications, we take into account the base positioning uncertainty and plan a se…
Khoury College of Computer Science（胡里计算机学院）；Northeastern University（东北大学）
Transporter Net is a recently proposed framework for pick and place that is able to learn good manipulation policies from a very few expert demonstrations. A key reason why Transporter Net is so sample efficient is that the model incorporates rotational equivariance into the pick module, i.e. the model immediately generalizes learned pick knowledge to objects presented in different orientations. This paper proposes a novel version of Transporter Net that is equivariant to both pick and place orientation. As a result, our model immediately generalizes place knowledge to different place orientat…
Robotic pick-and-place has been researched for a long time to cope with uncertainty of novel objects and changeable environments. Past works mainly focus on learning-based methods to achieve high precision. However, they have difficulty being generalized for the limitation of specified training models. To break through this drawback of learning-based approaches, we introduce a new perspective of similarity matching between novel objects and a known database based on category-association to achieve pick-and-place tasks with high accuracy and stabilization. We calculate the category name similar…
