Recent advances in deep reinforcement learning (RL) have demonstrated its potential to learn complex robotic manipulation tasks. However, RL still requires the robot to collect a large amount of real-world experience. To address this problem, recent works have proposed learning from expert demonstrations (LfD), particularly via inverse reinforcement learning (IRL), given its ability to achieve robust performance with only a small number of expert demonstrations. Nevertheless, deploying IRL on real robots is still challenging due to the large number of robot experiences it requires. This paper aims to address this scalability challenge with a robust, sample-efficient, and general meta-IRL algorithm, SQUIRL, that performs a new but related long-horizon task robustly given only a single video demonstration. First, this algorithm bootstraps the learning of a task encoder and a task-conditioned policy using behavioral cloning (BC). It then collects real-robot experiences and bypasses reward learning by directly recovering a Q-function from the combined robot and expert trajectories. Next, this algorithm uses the Q-function to re-evaluate all cumulative experiences collected by the robot to improve the policy quickly. In the end, the policy performs more robustly (90%+ success) than BC on new tasks while requiring no trial-and-errors at test time. Finally, our real-robot and simulated experiments demonstrate our algorithm's generality across different state spaces, action spaces, and vision-based manipulation tasks, e.g., pick-pour-place and pick-carry-drop.
University of Texas at Austin（德克萨斯大学奥斯汀分校）；Sony AI（索尼人工智能）
We tackle real-world long-horizon robot manipulation tasks through skill discovery. We present a bottom-up approach to learning a library of reusable skills from unsegmented demonstrations and use these skills to synthesize prolonged robot behaviors. Our method starts with constructing a hierarchical task structure from each demonstration through agglomerative clustering. From the task structures of multi-task demonstrations, we identify skills based on the recurring patterns and train goal-conditioned sensorimotor policies with hierarchical imitation learning. Finally, we train a meta control…
University of California at Berkeley（加州大学伯克利分校）；Technische Universität Berlin（柏林工业大学）
Dexterous multi-fingered hands can accomplish fine manipulation behaviors that are infeasible with simple robotic grippers. However, sophisticated multi-fingered hands are often expensive and fragile. Low-cost soft hands offer an appealing alternative to more conventional devices, but present considerable challenges in sensing and actuation, making them difficult to apply to more complex manipulation tasks. In this paper, we describe an approach to learning from demonstration that can be used to train soft robotic hands to perform dexterous manipulation tasks. Our method uses object-centric de…
DexCanvas: Bridging Human Demonstrations and Robot Learning for Dexterous Manipulation
DexRobot Co. Ltd.（德克斯机器人有限公司）；University of Michigan（密歇根大学）；Shanghai Jiao Tong University（上海交通大学）；Chongqing University（重庆大学）；East China University of Science and Technology（东华大学）
Imitation learning is a promising approach to help robots acquire dexterous manipulation capabilities without the need for a carefully-designed reward or a significant computational effort. However, existing imitation learning approaches require sophisticated data collection infrastructure and struggle to generalize beyond the training distribution. One way to address this limitation is to gather additional data that better represents the full operating conditions. In this work, we investigate characteristics of such additional demonstrations and their impact on performance. Specifically, we s…
EaDex: A Cross-Embodiment Dexterous Manipulation Framework from Low-Cost Demonstrations
Faculty of Robot Science and Engineering, Northeastern University（机器人科学与工程学院，东北大学）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究所，中国科学院）；School of Automation, Nanjing University of Information Science and Technology（自动化学院，南京信息科学技术大学）
Robot Deformable Object Manipulation via NMPC-generated Demonstrations in Deep Reinforcement Learning
Hubei Key Laboratory of Brain-inspired Intelligent Systems and the Key Laboratory of Image Processing and Intelligent Control, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（湖北脑启发智能系统重点实验室和图像处理与智能控制重点实验室，人工智能与自动化学院，华中科技大学）
Dexterous manipulation with anthropomorphic robot hands remains a challenging problem in robotics because of the high-dimensional state and action spaces and complex contacts. Nevertheless, skillful closed-loop manipulation is required to enable humanoid robots to operate in unstructured real-world environments. Reinforcement learning (RL) has traditionally imposed enormous interaction data requirements for optimizing such complex control problems. We introduce a new framework that leverages recent advances in GPU-based simulation along with the strength of imitation learning in guiding policy…
Vision-based learning methods provide promise for robots to learn complex manipulation tasks. However, how to generalize the learned manipulation skills to real-world interactions remains an open question. In this work, we study robotic manipulation skill learning from a single third-person view demonstration by using activity recognition and object detection in computer vision. To facilitate generalization across objects and environments, we propose to use a prior knowledge base in the form of a text corpus to infer the object to be interacted with in the context of a robot. We evaluate our a…
Stanford University（斯坦福大学）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）
Imitating human demonstrations is a promising approach to endow robots with various manipulation capabilities. While recent advances have been made in imitation learning and batch (offline) reinforcement learning, a lack of open-source human datasets and reproducible learning methods make assessing the state of the field difficult. In this paper, we conduct an extensive study of six offline learning algorithms for robot manipulation on five simulated and three real-world multi-stage manipulation tasks of varying complexity, and with datasets of varying quality. Our study analyzes the most crit…
While significant progress has been made on understanding hand-object interactions in computer vision, it is still very challenging for robots to perform complex dexterous manipulation. In this paper, we propose a new platform and pipeline DexMV (Dexterous Manipulation from Videos) for imitation learning. We design a platform with: (i) a simulation system for complex dexterous manipulation tasks with a multi-finger robot hand and (ii) a computer vision system to record large-scale demonstrations of a human hand conducting the same tasks. In our novel pipeline, we extract 3D hand and object pos…
University of Lincoln（林肯大学）；Lincoln Institute for Agri-Food Technology（林肯农业食品技术研究所）
Robots learning a new manipulation task from a small amount of demonstrations are increasingly demanded in different workspaces. A classifier model assessing the quality of actions can predict the successful completion of a task, which can be used by intelligent agents for action-selection. This paper presents a novel classifier that learns to classify task completion only from a few demonstrations. We carry out a comprehensive comparison of different neural classifiers, e.g. fully connected-based, fully convolutional-based, sequence2sequence-based, and domain adaptation-based classification. …
University of Toronto Institute for Aerospace Studies（多伦多大学航空航天研究所）；Vector Institute（矢量研究所）
Learned visuomotor policies have shown considerable success as an alternative to traditional, hand-crafted frameworks for robotic manipulation. Surprisingly, an extension of these methods to the multiview domain is relatively unexplored. A successful multiview policy could be deployed on a mobile manipulation platform, allowing the robot to complete a task regardless of its view of the scene. In this work, we demonstrate that a multiview policy can be found through imitation learning by collecting data from a variety of viewpoints. We illustrate the general applicability of the method by learn…
Learning from demonstrations is a promising paradigm for transferring knowledge to robots. However, learning mobile manipulation tasks directly from a human teacher is a complex problem as it requires learning models of both the overall task goal and of the underlying actions. Additionally, learning from a small number of demonstrations often introduces ambiguity with respect to the intention of the teacher, making it challenging to commit to one model for generalizing the task to new settings. In this paper, we present an approach to learning flexible mobile manipulation action models and tas…
DexRoam: Learning Mobile Bimanual Dexterous Manipulation from Egocentric Whole-Body Human Demonstrations
The Hong Kong University of Science and Technology（香港科技大学）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University（北京大学计算机学院多媒体信息处理国家重点实验室）；Beihang University（北京航空航天大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
Mastering robotic manipulation skills through reinforcement learning (RL) typically requires the design of shaped reward functions. Recent developments in this area have demonstrated that using sparse rewards, i.e. rewarding the agent only when the task has been successfully completed, can lead to better policies. However, state-action space exploration is more difficult in this case. Recent RL approaches to learning with sparse rewards have leveraged high-quality human demonstrations for the task, but these can be costly, time consuming or even impossible to obtain. In this paper, we propose …
Learning actions from human demonstration is an emerging trend for designing intelligent robotic systems, which can be referred as video to command. The performance of such approach highly relies on the quality of video captioning. However, the general video captioning methods focus more on the understanding of the full frame, lacking of consideration on the specific object of interests in robotic manipulations. We propose a novel deep model to learn actions from human demonstration video for robotic manipulation. It consists of two deep networks, grasp detection network (GNet) and video capti…
Learning Visual Quadrupedal Loco-Manipulation from Demonstrations
Shanghai Qi Zhi Institute（上海期智研究院）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；University of California, Berkeley（加州大学伯克利分校）；Tsinghua University（清华大学）
University of Washington（华盛顿大学）；OpenAI；University of California Berkeley（加州大学伯克利分校）；Istituto Italiano di Tecnologia（意大利理工学院）
Dexterous multi-fingered hands are extremely versatile and provide a generic way to perform a multitude of tasks in human-centric environments. However, effectively controlling them remains challenging due to their high dimensionality and large number of potential contacts. Deep reinforcement learning (DRL) provides a model-agnostic approach to control complex dynamical systems, but has not been shown to scale to high-dimensional dexterous manipulation. Furthermore, deployment of DRL on physical systems remains challenging due to sample inefficiency. Consequently, the success of DRL in robotic…
There is a large variety of objects and appliances in human environments, such as stoves, coffee dispensers, juice extractors, and so on. It is challenging for a roboticist to program a robot for each of these object types and for each of their instantiations. In this work, we present a novel approach to manipulation planning based on the idea that many household objects share similarly-operated object parts. We formulate the manipulation planning as a structured prediction problem and design a deep learning model that can handle large noise in the manipulation demonstrations and learns featur…
Dexterous manipulation with a multi-finger hand is one of the most challenging problems in robotics. While recent progress in imitation learning has largely improved the sample efficiency compared to Reinforcement Learning, the learned policy can hardly generalize to manipulate novel objects, given limited expert demonstrations. In this paper, we propose to learn dexterous manipulation using large-scale demonstrations with diverse 3D objects in a category, which are generated from a human grasp affordance model. This generalizes the policy to novel object instances within the same category. To…
CoBT: Collaborative Programming of Behaviour Trees from One Demonstration for Robot Manipulation
Technological University Dublin（都柏林理工大学）；Irish Manufacturing Research（爱尔兰制造研究院）；Atlantic Technological University（大西洋理工大学）；University of Modena and Reggio Emilia（摩德纳-雷焦艾米利亚大学）；Trinity College Dublin（都柏林圣三一大学）；ADAPT Research Centre（ADAPT研究中心）
DexViTac: Collecting Human Visuo-Tactile-Kinematic Demonstrations for Contact-Rich Dexterous Manipulation
State Key Laboratory of Intelligent Manufacturing Equipment and Technology（智能制造装备与技术国家重点实验室）；Huazhong University of Science and Technology（华中科技大学）；Wuhan Huaweike Intelligent Technology Co., Ltd.（武汉华为凯科技有限公司）
Should We Learn Contact-Rich Manipulation Policies from Sampling-Based Planners?
New York University（纽约大学）；Boston Dynamics AI Institute（波士顿动力AI研究院）；Georgia Institute of Technology（佐治亚理工学院）；Cornell University（康奈尔大学）；Artificial and Natural Intelligence Toulouse Institute（图卢兹人工智能与自然智能研究所）
SkillWeave: Weaving Heterogeneous Demonstrations into Long-Horizon Manipulation Skills
Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人学院）；Keio University（庆应义塾大学）；College of Engineering, University of California, Berkeley（加州大学伯克利分校工程学院）；Bosch Center for Artificial Intelligence（博世人工智能中心）
Beyond Viewpoint Generalization: What Multi-View Demonstrations Offer and How to Synthesize Them for Robot Manipulation?
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Shenzhen University（深圳大学）；Beijing Jiaotong University（北京交通大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
