Multi-step manipulation tasks in unstructured environments are extremely challenging for a robot to learn. Such tasks interlace high-level reasoning that consists of the expected states that can be attained to achieve an overall task and low-level reasoning that decides what actions will yield these states. We propose a model-free deep reinforcement learning method to learn multi-step manipulation tasks. We introduce a Robotic Manipulation Network (RoManNet), which is a vision-based model architecture, to learn the action-value functions and predict manipulation action candidates. We define a Task Progress based Gaussian (TPG) reward function that computes the reward based on actions that lead to successful motion primitives and progress towards the overall task goal. To balance the ratio of exploration/exploitation, we introduce a Loss Adjusted Exploration (LAE) policy that determines actions from the action candidates according to the Boltzmann distribution of loss estimates. We demonstrate the effectiveness of our approach by training RoManNet to learn several challenging multi-step robotic manipulation tasks in both simulation and real-world. Experimental results show that our method outperforms the existing methods and achieves state-of-the-art performance in terms of success rate and action efficiency. The ablation studies show that TPG and LAE are especially beneficial for tasks like multiple block stacking. Code is available at: https://github.com/skumra/romannet
The past decade has witnessed the tremendous successes of machine learning techniques in the supervised learning paradigm, where there is a clear demarcation between training and testing. In the supervised learning paradigm, learning is inherently passive, seeking to distill human-provided supervision in large-scale datasets into high capacity models. Following these successes, machine learning researchers have looked beyond this paradigm and became interested in tasks that are more dynamic. To them, robotics serve as an excellent test-bed, for the challenges of robotics break many of the assu…
Robotic control tasks are often first run in simulation for the purposes of verification, debugging and data augmentation. Many methods exist to specify what task a robot must complete, but few exist to specify what range of environments a user expects such tasks to be achieved in. ProbRobScene is a probabilistic specification language for describing robotic manipulation environments. Using the language, a user need only specify the relational constraints that must hold between objects in a scene. ProbRobScene will then automatically generate scenes which conform to this specification. By comb…
Queensland University of Technology (QUT)（昆士兰科技大学）；Data61/CSIRO（Data61/联邦科学与工业研究组织）；Australian Centre for Robotic Vision (ACRV)（澳大利亚机器人视觉中心）
We quantify the accuracy of various simulators compared to a real world robotic reaching and interaction task. Simulators are used in robotics to design solutions for real world hardware without the need for physical access. The `reality gap' prevents solutions developed or learnt in simulation from performing well, or at at all, when transferred to real-world hardware. Making use of a Kinova robotic manipulator and a motion capture system, we record a ground truth enabling comparisons with various simulators, and present quantitative data for various manipulation-oriented robotic tasks. We sh…
RGBManip: Monocular Image-based Robotic Manipulation through Active Object Pose Estimation
Hyperplane Lab, School of CS, Peking University（北京大学计算机学院超平面实验室）；National Key Laboratory for Multimedia Information Processing（多媒体信息处理国家重点实验室）；Beijing Academy of Artificial Intelligence (BAAI)（北京人工智能研究院）；Department of Computer Science and Engineering, The Chinese University of Hong Kong（香港中文大学计算机科学与工程学系）
Referring expressions are commonly used when referring to a specific target in people's daily dialogue. In this paper, we develop a novel task of audio-visual grounding referring expression for robotic manipulation. The robot leverages both the audio and visual information to understand the referring expression in the given manipulation instruction and the corresponding manipulations are implemented. To solve the proposed task, an audio-visual framework is proposed for visual localization and sound recognition. We have also established a dataset which contains visual data, auditory data and ma…
Dublin City University（都柏林城市大学）；Insight SFI Research Centre for Data Analytics（Insight SFI数据分析研究中心）；University College Dublin（都柏林大学学院）
Robotic manipulation and control has increased in importance in recent years. However, state of the art techniques still have limitations when required to operate in real world applications. This paper explores Hindsight Experience Replay both in simulated and real environments, highlighting its weaknesses and proposing reinforcement-learning based alternatives based on reward and goal shaping. Additionally, several research questions are identified along with potential research directions that could be explored to tackle those questions.
Learning Smooth and Robust Space Robotic Manipulation of Dynamic Target via Inter-frame Correlation
Research Center for Intelligent Robotics, School of Astronautics, Northwestern Polytechnical University（西北工业大学航天学院智能机器人研究中心）；School of Automation, Northwestern Polytechnical University（西北工业大学自动化学院）；School of Aeronautics and Astronautics, Sichuan University（四川大学空天科学与工程学院）
High resolution tactile sensors are often bulky and have shape profiles that make them awkward for use in manipulation. This becomes important when using such sensors as fingertips for dexterous multi-fingered hands, where boxy or planar fingertips limit the available set of smooth manipulation strategies. High resolution optical based sensors such as GelSight have until now been constrained to relatively flat geometries due to constraints on illumination geometry.Here, we show how to construct a rounded fingertip that utilizes a form of light piping for directional illumination. Our sensors c…
Data61, Commonwealth Scientific and Industrial Research Organisation (CSIRO)（联邦科学与工业研究组织（CSIRO）Data61研究所）；Queensland University of Technology (QUT)（昆士兰科技大学（QUT））
We present a benchmark to facilitate simulated manipulation; an attempt to overcome the obstacles of physical benchmarks through the distribution of a real world, ground truth dataset. Users are given various simulated manipulation tasks with assigned protocols having the objective of replicating the real world results of a recorded dataset. The benchmark comprises of a range of metrics used to characterise the successes of submitted environments whilst providing insight into their deficiencies. We apply our benchmark to two simulation environments, PyBullet and V-Rep, and publish the results.…
We present BulletArm, a novel benchmark and learning-environment for robotic manipulation. BulletArm is designed around two key principles: reproducibility and extensibility. We aim to encourage more direct comparisons between robotic learning methods by providing a set of standardized benchmark tasks in simulation alongside a collection of baseline algorithms. The framework consists of 31 different manipulation tasks of varying difficulty, ranging from simple reaching and picking tasks to more realistic tasks such as bin packing and pallet stacking. In addition to the provided tasks, BulletAr…
A neural network based flexible object manipulation system for a humanoid robot on FPGA is proposed. Although the manipulations of flexible objects using robots attract ever increasing attention since these tasks are the basic and essential activities in our daily life, it has been put into practice only recently with the help of deep neural networks. However such systems have relied on GPU accelerators, which cannot be implemented into the space limited robotic body. Although field programmable gate arrays (FPGAs) are known to be energy efficient and suitable for embedded systems, the model s…
University of Washington（华盛顿大学）；Google, Inc.（谷歌公司）；University of California Berkeley（加州大学伯克利分校）
In this paper, we examine the problem of robotic manipulation of granular media. We evaluate multiple predictive models used to infer the dynamics of scooping and dumping actions. These models are evaluated on a task that involves manipulating the media in order to deform it into a desired shape. Our best performing model is based on a highly-tailored convolutional network architecture with domain-specific optimizations, which we show accurately models the physical interaction of the robotic scoop with the underlying media. We empirically demonstrate that explicitly predicting physical mechani…
In this paper, we propose a new action planning approach to automatically pack long linear elastic objects into common-size boxes with a bimanual robotic system. For that, we developed a hybrid geometric model to handle large-scale occlusions combining an online vision-based method and an offline reference template. Then, a reference point generator is introduced to automatically plan the reference poses for the predesigned action primitives. Finally, an action planner integrates these components enabling the execution of high-level behaviors and the accomplishment of packing manipulation task…
Osaka University（大阪大学）；University of California, Santa Cruz（加州大学圣克鲁兹分校）
This paper develops a robotic manipulation planner for human-robot collaborative assembly. Unlike previous methods which study an independent and fully AI-equipped autonomous system, this paper explores the subtask distribution between a robot and a human and studies a human-in-the-loop robotic system for collaborative assembly. The system distributes the subtasks of an assembly to robots and humans by exploiting their advantages and avoiding their disadvantages. The robot in the system will work on pick-and-place tasks and provide workpieces to humans. The human collaborator will work on fine…
Istituto Italiano di Tecnologia（意大利理工学院）；Università Cattolica del Sacro Cuore（圣心天主教大学）
In recent years Sim2Real approaches have brought great results to robotics. Techniques such as model-based learning or domain randomization can help overcome the gap between simulation and reality, but in some situations simulation accuracy is still needed. An example is agricultural robotics, which needs detailed simulations, both in terms of dynamics and visuals. However, simulation software is still not capable of such quality and accuracy. Current Sim2Real techniques are helpful in mitigating the problem, but for these specific tasks they are not enough.
Air Traffic Management Research Institute（空中交通管理研究院）；Singapore Centre for 3D Printing（新加坡3D打印中心）；Nanyang Technological University（南洋理工大学）
This paper considers the problem of manipulating a uniformly rotating chain: the chain is rotated at a constant angular speed around a fixed axis using a robotic manipulator. Manipulation is quasi-static in the sense that transitions are slow enough for the chain to be always in "rotational" equilibrium. The curve traced by the chain in a rotating plane -- its shape function -- can be determined by a simple force analysis, yet it possesses complex multi-solutions behavior typical of non-linear systems. We prove that the configuration space of the uniformly rotating chain is homeomorphic to a t…
Royal Institute of Technology（皇家理工学院）；ASEA Brown Boveri (ABB) Corporate Research（ASEA布朗勃法瑞（ABB）公司研究部）
Reinforcement Learning (RL) of robotic manipulation skills, despite its impressive successes, stands to benefit from incorporating domain knowledge from control theory. One of the most important properties that is of interest is control stability. Ideally, one would like to achieve stability guarantees while staying within the framework of state-of-the-art deep RL algorithms. Such a solution does not exist in general, especially one that scales to complex manipulation tasks. We contribute towards closing this gap by introducing $\textit{normalizing-flow}$ control structure, that can be deploye…
ICube Lab Robotics Department Strasbourg University UMR 7357 CNRS（斯特拉斯堡大学ICube实验室机器人系 UMR 7357 CNRS）；ICube Lab UMR 7357 Centre National de la Recherche Scientifique CNRS（ICube实验室 UMR 7357 法国国家科学研究中心）
Deep learning has provided new ways of manipulating, processing and analyzing data. It sometimes may achieve results comparable to, or surpassing human expert performance, and has become a source of inspiration in the era of artificial intelligence. Another subfield of machine learning named reinforcement learning, tries to find an optimal behavior strategy through interactions with the environment. Combining deep learning and reinforcement learning permits resolving critical issues relative to the dimensionality and scalability of data in tasks with sparse reward signals, such as robotic mani…
University of South Florida（南佛罗里达大学）；National Institute of Standards and Technology (NIST)（美国国家标准与技术研究院）；German Aerospace Center (DLR)（德国航空航天中心）；Worcester Polytechnic Institute (WPI)（伍斯特理工学院）
This paper discusses recent research progress in robotic grasping and manipulation in the light of the latest Robotic Grasping and Manipulation Competitions (RGMCs). We first provide an overview of past benchmarks and competitions related to the robotics manipulation field. Then, we discuss the methodology behind designing the manipulation tasks in RGMCs. We provide a detailed analysis of key challenges for each task and identify the most difficult aspects based on the competing teams' performance in recent years. We believe that such an analysis is insightful to determine the future research …
Cognitive Robotics, TU Delft（代尔夫特理工大学认知机器人）；Honda Research Institute, Europe（本田研究院欧洲）；LIRMM - Université de Montpellier CNRS（蒙彼利埃大学 CNRS LIRMM）；COSMER laboratory EA 7398, Universié de Toulon（土伦大学 COSMER 实验室 EA 7398）；The Hong Kong Polytechnic University（香港理工大学）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；University of Michigan（密歇根大学）；Università degli Studi di Napoli Federico II（那不勒斯费德里科二世大学）；Osaka University（大阪大学）；National Institute of AIST（产业技术综合研究所）；Tsinghua University（清华大学）；The University of Hong Kong（香港大学）；Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）
Deformable object manipulation (DOM) is an emerging research problem in robotics. The ability to manipulate deformable objects endows robots with higher autonomy and promises new applications in the industrial, services, and healthcare sectors. However, compared to rigid object manipulation, the manipulation of deformable objects is considerably more complex, and is still an open research problem. Addressing DOM challenges demand breakthroughs in almost all aspects of robotics, namely hardware design, sensing, (deformation) modeling, planning, and control. In this article, we review recent adv…
DORA: Object Affordance-Guided Reinforcement Learning for Dexterous Robotic Manipulation
TAMS (Technical Aspects of Multimodal Systems), Department of Informatics, Universität Hamburg（技术多模态系统（TAMS），信息学院，汉堡大学）；Agile Robots AG（敏捷机器人公司）；Technical University of Munich（慕尼黑技术大学）
