Residual Compliant Policy: Learning Compliant Policies from Few Demonstrations
University of Pennsylvania（宾夕法尼亚大学）；Technical University of Darmstadt（达姆施塔特工业大学）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；German Research Center for AI (DFKI)（德国人工智能研究中心）；Robotics Institute Germany (RIG)（德国机器人研究所）
Chunyu Jiang, Yichen Cai, Shuai Li, Wenbin Sun, João Carvalho, Kostas Daniilidis, Oleg Arenz, Cristiana de Farias, Jan Peters
Contact-rich manipulation requires a robot to be precise in free space to establish the intended contacts, and compliant once in contact to avoid excessive forces. Imitation learning has proven effective for simpler manipulation tasks, but the compliance underlying a demonstration is not directly observable and is therefore difficult to infer from demonstration data alone. We present a framework that makes compliance explicit at every stage. During data collection, the operator teleoperates the real robot while a synchronized digital twin visualizes the error between the commanded reference and the achieved pose, making the current compliance visible, and selects between two discrete compliance modes. To generalize across object poses, we combine Laplacian editing, which retargets trajectories efficiently but is not physics-consistent during contact, with simulation-based trajectory optimization, which enforces physical plausibility but offers no manual guidance. Finally, a residual reinforcement learning policy refines the interaction forces by adjusting the reference pose of the underlying admittance controller. We evaluate on three flipping and pushing tasks on a Franka FR3 based on a total of 7 demonstrations across tasks. Our method is successful on all 40 out-of-distribution trials, compared to 33/40 and 24/40 for two baselines, with our base policy alone reaching 35/40.
This paper proposes a method to learn from human demonstration compliant contact motions, which take advantage of interaction forces between workpieces to align them, even when contact force may occur from different directions on different instances of reproduction. To manage the uncertainty in unstructured conditions, the motions learned with our method can be reproduced with an impedance controller. Learning from Demonstration is used because the planning of compliant motions in 3-D is computationally intractable. The proposed method will learn an individual compliant motion, many of which c…
We present a novel method for learning from demonstration 6-D tasks that can be modeled as a sequence of linear motions and compliances. The focus of this paper is the learning of a single linear primitive, many of which can be sequenced to perform more complex tasks. The presented method learns from demonstrations only, without any prior information, how to take advantage of mechanical gradients in in-contact tasks, such as assembly, both for translations and rotations. The method assumes there exists a desired linear direction in 6-D which, if followed by the manipulator, leads the robot's e…
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Army Research Laboratory（陆军研究实验室）；Google（谷歌）
Social navigation is the capability of an autonomous agent, such as a robot, to navigate in a 'socially compliant' manner in the presence of other intelligent agents such as humans. With the emergence of autonomously navigating mobile robots in human populated environments (e.g., domestic service robots in homes and restaurants and food delivery robots on public sidewalks), incorporating socially compliant navigation behaviors on these robots becomes critical to ensuring safe and comfortable human robot coexistence. To address this challenge, imitation learning is a promising framework, since …
University of Oulu（奥卢大学）；Aalto University（阿尔托大学）；Idiap Research Institute（Idiap研究所）
In this paper we show how different choices regarding compliance affect a dual-arm assembly task. In addition, we present how the compliance parameters can be learned from a human demonstration. Compliant motions can be used in assembly tasks to mitigate pose errors originating from, for example, inaccurate grasping. We present analytical background and accompanying experimental results on how to choose the center of compliance to enhance the convergence region of an alignment task. Then we present the possible ways of choosing the compliant axes for accomplishing alignment in a scenario where…
DemoSpeedup: Accelerating Visuomotor Policies via Entropy-Guided Demonstration Acceleration
Shanghai Qi Zhi Institute（上海启智研究院）；Tsinghua University IIIS（清华大学智能科学研究院）；Shanghai AI Lab（上海人工智能实验室）；University of Electronic Science and Technology of China（电子科技大学）
Nara Institute of Science and Technology（奈良先端科学技术大学院大学）
Generative Adversarial Imitation Learning (GAIL) can learn policies without explicitly defining the reward function from demonstrations. GAIL has the potential to learn policies with high-dimensional observations as input, e.g., images. By applying GAIL to a real robot, perhaps robot policies can be obtained for daily activities like washing, folding clothes, cooking, and cleaning. However, human demonstration data are often imperfect due to mistakes, which degrade the performance of the resulting policies. We address this issue by focusing on the following features: 1) many robotic tasks are …
VOLT: Vision and Language Trajectory Segmentation for Faster-than-Demonstration Policies
Collab , Dept. of Mechanical Engineering, Virginia Tech, Blacksburg, VA 24061（机械工程系，弗吉尼亚理工学院，布莱克斯堡，VA 24061）；Mitsubishi Electric Research Laboratories ( MERL ), Cambridge, MA 02139（三菱电机研究实验室（MERL），剑桥，MA 02139）
Aalto University（阿尔托大学）；Tampere University of Technology（坦佩雷理工大学）
This paper presents, for the first time, a method for learning in-contact tasks from a teleoperated demonstration with a hydraulic manipulator. Due to the use of extremely powerful hydraulic manipulator, a force-reflected bilateral teleoperation is the most reasonable method of giving a human demonstration. An advanced subsystem-dynamic-based control design framework, virtual decomposition control (VDC), is used to design a stability-guaranteed controller for the teleoperation system, while taking into account the full nonlinear dynamics of the master and slave manipulators. The use of fragile…
Decompose and Reorganize: Planning with Primitives and Visuomotor Policies Learned from Demonstrations
The University of Hong Kong（香港大学）；JD.com (JingDong)（京东）；Nanyang Technological University (NTU)（南洋理工大学）；Southern University of Science and Technology (SUSTech)（南方科技大学）；Huawei Technologies（华为技术有限公司）；The Chinese University of Hong Kong (CUHK)（香港中文大学）
ReWiND: Language-Guided Rewards Teach Robot Policies without New Demonstrations
Thomas Lord Department of Computer Science, University of Southern California（南加州大学计算机科学系）；Amazon Robotics（亚马逊机器人技术）；Kim Jaechul School of Artificial Intelligence, KAIST（韩国科学技术院人工智能学院）
Humans demonstrate a variety of interesting behavioral characteristics when performing tasks, such as selecting between seemingly equivalent optimal actions, performing recovery actions when deviating from the optimal trajectory, or moderating actions in response to sensed risks. However, imitation learning, which attempts to teach robots to perform these same tasks from observations of human demonstrations, often fails to capture such behavior. Specifically, commonly used learning algorithms embody inherent contradictions between the learning assumptions (e.g., single optimal action) and actu…
Nara Institute of Science and Technology（奈良先端科学技术大学院大学）
Robust imitation learning using disturbance injections overcomes issues of limited variation in demonstrations. However, these methods assume demonstrations are optimal, and that policy stabilization can be learned via simple augmentations. In real-world scenarios, demonstrations are often of diverse-quality, and disturbance injection instead learns sub-optimal policies that fail to replicate desired behavior. To address this issue, this paper proposes a novel imitation learning framework that combines both policy robustification and optimal demonstration learning. Specifically, this combinato…
Learning from demonstration (LfD) is an intuitive framework allowing non-expert users to easily (re-)program robots. However, the quality and quantity of demonstrations have a great influence on the generalization performances of LfD approaches. In this paper, we introduce a novel active learning framework in order to improve the generalization capabilities of control policies. The proposed approach is based on the epistemic uncertainties of Bayesian Gaussian mixture models (BGMMs). We determine the new query point location by optimizing a closed-form information-density cost based on the quad…
Idiap Research Institute（Idiap研究院）；University of Bern（伯尔尼大学）
In learning from demonstrations, it is often desirable to adapt the behavior of the robot as a function of the variability retrieved from human demonstrations and the (un)certainty encoded in different parts of the task. In this paper, we propose a novel multi-output Gaussian process (MOGP) based on Gaussian mixture regression (GMR). The proposed approach encapsulates the variability retrieved from the demonstrations in the covariance of the MOGP. Leveraging the generative nature of GP models, our approach can efficiently modulate trajectories towards new start-, via- or end-points defined by …
From a Single Demonstration to a General Policy for Contact-Rich Manipulation
Robotics and Biology Laboratory, Technische Universität Berlin（技术大学柏林机器人与生物学实验室）；Science of Intelligence, Research Cluster of Excellence, Berlin（智能科学，卓越研究集群，柏林）；Robotics Institute Germany（德国机器人研究所）
In this work we propose a novel end-to-end imitation learning approach which combines natural language, vision, and motion information to produce an abstract representation of a task, which in turn is used to synthesize specific motion controllers at run-time. This multimodal approach enables generalization to a wide variety of environmental conditions and allows an end-user to direct a robot policy through verbal communication. We empirically validate our approach with an extensive set of simulations and show that it achieves a high task success rate over a variety of conditions while remaini…
In the learning from demonstration (LfD) paradigm, understanding and evaluating the demonstrated behaviors plays a critical role in extracting control policies for robots. Without this knowledge, a robot may infer incorrect reward functions that lead to undesirable or unsafe control policies. Recent work has proposed an LfD framework where a user provides a set of formal task specifications to guide LfD, to address the challenge of reward shaping. However, in this framework, specifications are manually ordered in a performance graph (a partial order that specifies relative importance between t…
This work aims to learn how to perform complex robot manipulation tasks that are composed of several, consecutively executed low-level sub-tasks, given as input a few visual demonstrations of the tasks performed by a person. The sub-tasks consist of moving the robot's end-effector until it reaches a sub-goal region in the task space, performing an action, and triggering the next sub-task when a pre-condition is met. Most prior work in this domain has been concerned with learning only low-level tasks, such as hitting a ball or reaching an object and grasping it. This paper describes a new neura…
Department of Computer Science, ETH Z\"urich, Z\"urich, Switzerland；Max Planck Institute for Intelligent Systems, T\"ubingen, Germany；University of T\"ubingen, T\"ubingen, Germany
Training-Free Imitation Learning with Closed-Form Diffusion Policies
Australian Center for Robotics, ARIAM Hub, and School of Aerospace, Mechanical and Mechatronic Engineering University of Sydney（澳大利亚机器人中心、ARIAM中心和悉尼大学航空航天、机械与机电工程学院）
Learning-from-demonstrations is an emerging paradigm to obtain effective robot control policies for complex tasks via reinforcement learning without the need to explicitly design reward functions. However, it is susceptible to imperfections in demonstrations and also raises concerns of safety and interpretability in the learned control policies. To address these issues, we use Signal Temporal Logic to evaluate and rank the quality of demonstrations. Temporal logic-based specifications allow us to create non-Markovian rewards, and also define interesting causal dependencies between tasks such a…
School of Electrical Engineering, Aalto University（阿尔托大学电气工程学院）
This paper proposes an approach for segmenting a task consisting of compliant motions into phases, learning a primitive for each segmented phase of the task, and reproducing the task by sequencing primitives online based on the learned model. As compliant motions can "probe" the environment, using the interaction between the robot and the environment to detect phase transitions can make the transitions less prone to positional errors. This intuition leads us to model a task with a non-homogeneous Hidden Markov Model (HMM), wherein hidden phase transition probabilities depend on the interaction…
CRISP -- Compliant ROS2 Controllers for Learning-Based Manipulation Policies and Teleoperation
Technical University of Munich（慕尼黑技术大学）；TUM School of Computation, Information and Technology（TUM计算、信息与技术学院）；Department of Computer Engineering（计算机工程系）；Learning Systems and Robotics Lab（学习系统与机器人实验室）；TUM Munich Institute of Robotics and Machine Intelligence (MIRMI)（慕尼黑技术大学机器人与人工智能研究所（MIRMI））
Imitation learning is an effective and safe technique to train robot policies in the real world because it does not depend on an expensive random exploration process. However, due to the lack of exploration, learning policies that generalize beyond the demonstrated behaviors is still an open challenge. We present a novel imitation learning framework to enable robots to 1) learn complex real world manipulation tasks efficiently from a small number of human demonstrations, and 2) synthesize new behaviors not contained in the collected demonstrations. Our key insight is that multi-task domains of…
Imitation learning enables robots to learn from demonstrations. Previous imitation learning algorithms usually assume access to optimal expert demonstrations. However, in many real-world applications, this assumption is limiting. Most collected demonstrations are not optimal or are produced by an agent with slightly different dynamics. We therefore address the problem of imitation learning when the demonstrations can be sub-optimal or be drawn from agents with varying dynamics. We develop a metric composed of a feasibility score and an optimality score to measure how useful a demonstration is …
Shanghai Qi Zhi Institute（上海期智研究院）；Tsinghua University（清华大学）；Shanghai AI Lab（上海人工智能实验室）；Massachusetts Institute of Technology（麻省理工学院）
Visual imitation learning provides an effective framework to learn skills from demonstrations. However, the quality of the provided demonstrations usually significantly affects the ability of an agent to acquire desired skills. Therefore, the standard visual imitation learning assumes near-optimal demonstrations, which are expensive or sometimes prohibitive to collect. Previous works propose to learn from noisy demonstrations; however, the noise is usually assumed to follow a context-independent distribution such as a uniform or gaussian distribution. In this paper, we consider another crucial…
Recent research demonstrated that it is feasible to end-to-end train multi-task deep visuomotor policies for robotic manipulation using variations of learning from demonstration (LfD) and reinforcement learning (RL). In this paper, we extend the capabilities of end-to-end LfD architectures to object manipulation in clutter. We start by introducing a data augmentation procedure called Accept Synthetic Objects as Real (ASOR). Using ASOR we develop two network architectures: implicit attention ASOR-IA and explicit attention ASOR-EA. Both architectures use the same training data (demonstrations in…
ReDex: Repairing Sim-to-Real Dexterous Policies by Finger-Level Compliant Interaction
Amazon（亚马逊）；Duke University（杜克大学）；University of Maryland, College Park（马里兰大学帕克分校）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）；University of Michigan（密歇根大学）
Georgia Institute of Technology（佐治亚理工学院）；FAIR, Meta AI（Meta人工智能研究院（FAIR））
We study ObjectGoal Navigation -- where a virtual robot situated in a new environment is asked to navigate to an object. Prior work has shown that imitation learning (IL) using behavior cloning (BC) on a dataset of human demonstrations achieves promising results. However, this has limitations -- 1) BC policies generalize poorly to new states, since the training mimics actions not their consequences, and 2) collecting demonstrations is expensive. On the other hand, reinforcement learning (RL) is trivially scalable, but requires careful reward engineering to achieve desirable behavior. We presen…
