Manipulation skills involving contact and friction are inherent to many robotics tasks. Using the class of motor primitives for peg-in-hole like insertions, we study how robots can learn such skills. Dynamic Movement Primitives (DMP) are a popular way of extracting such policies through behaviour cloning (BC) but can struggle in the context of insertion. Policy adaptation strategies such as residual learning can help improve the overall performance of policies in the context of contact-rich manipulation. However, it is not clear how to best do this with DMPs. As a result, we consider several possible ways for adapting a DMP formulation and propose ``residual Learning from Demonstration`` (rLfD), a framework that combines DMPs with Reinforcement Learning (RL) to learn a residual correction policy. Our evaluations suggest that applying residual learning directly in task space and operating on the full pose of the robot can significantly improve the overall performance of DMPs. We show that rLfD offers a gentle to the joints solution that improves the task success and generalisation of DMPs \rb{and enables transfer to different geometries and frictions through few-shot task adaptation}. The proposed framework is evaluated on a set of tasks. A simulated robot and a physical robot have to successfully insert pegs, gears and plugs into their respective sockets. Other material and videos accompanying this paper are provided at https://sites.google.com/view/rlfd/.
Learning from demonstration (LfD) is a powerful learning method to enable a robot to infer how to perform a task given one or more human demonstrations of the desired task. By learning from end-user demonstration rather than requiring that a domain expert manually programming each skill, robots can more readily be applied to a wider range of real-world applications. Writing robots, as one application of LfD, has become a challenging research topic due to the complexity of human handwriting trajectories. In this paper, we introduce a generalized handwriting-learning system for a physical robot …
BT-TL-DMPs: A Novel Robot TAMP Framework Combining Behavior Tree, Temporal Logic and Dynamical Movement Primitives
Institute of Robotics and Automatic Information Systems, College of Artificial Intelligence, Nankai University（机器人与自动信息系统研究所，人工智能学院，南开大学）；Tianjin Key Laboratory of Intelligent Robotics, Nankai University（智能机器人重点实验室，南开大学）；College of Artificial Intelligence, Nankai University（人工智能学院，南开大学）；Department of Computing, The Hong Kong Polytechnic University（计算学院，香港理工大学）
Robotic Grinding Skills Learning Based on Geodesic Length Dynamic Motion Primitives
State Key Laboratory of Intelligent Manufacturing Equipment and Technology, Department of Mechanical Science and Engineering, Huazhong University of Science and Technology（智能制造装备与技术国家重点实验室，机械科学与工程学院，华中科技大学）
University of Edinburgh（爱丁堡大学）；Monash University（莫纳什大学）
Dynamic Movement Primitives (DMPs) offer great versatility for encoding, generating and adapting complex end-effector trajectories. DMPs are also very well suited to learning manipulation skills from human demonstration. However, the reactive nature of DMPs restricts their applicability for tool use and object manipulation tasks involving non-holonomic constraints, such as scalpel cutting or catheter steering. In this work, we extend the Cartesian space DMP formulation by adding a coupling term that enforces a pre-defined set of non-holonomic constraints. We obtain the closed-form expression f…
University of Southern California（南加州大学）；Indian Institute of Technology, Kharagpur（印度理工学院卡拉格普尔分校）；Adobe MDSR
Humans excel at learning long-horizon tasks from demonstrations augmented with textual commentary, as evidenced by the burgeoning popularity of tutorial videos online. Intuitively, this capability can be separated into 2 distinct subtasks - first, dividing a long-horizon demonstration sequence into semantically meaningful events; second, adapting such events into meaningful behaviors in one's own environment. Here, we present Video2Skill (V2S), which attempts to extend this capability to artificial agents by allowing a robot arm to learn from human cooking videos. We first use sequence-to-sequ…
DFL-TORO: A One-Shot Demonstration Framework for Learning Time-Optimal Robotic Manufacturing Tasks
University of Luxembourg（卢森堡大学）；Interdisciplinary Centre for Security, Reliability, and Trust (SnT)（安全、可靠性与跨学科信任中心（SnT））；Faculty of Science, Technology, and Medicine（科学、技术与医学学院）
Agile Robots AG（敏捷机器人公司）；Fraunhofer IPK（弗劳恩霍夫生产设备与设计技术研究所）；Universität Hamburg（汉堡大学）；Intel Asia-Pacific Research & Development Ltd.（英特尔亚太研发有限公司）
Dynamic movement primitives (DMPs) allow complex position trajectories to be efficiently demonstrated to a robot. In contact-rich tasks, where position trajectories alone may not be safe or robust over variation in contact geometry, DMPs have been extended to include force trajectories. However, different task phases or degrees of freedom may require the tracking of either position or force -- e.g., once contact is made, it may be more important to track the force demonstration trajectory in the contact direction. The robot impedance balances between following a position or force reference tra…
DynaMimicGen: A Data Generation Framework for Robot Learning of Dynamic Tasks
Institute of Systems and Technologies for Sustainable Production (ISTePS)（可持续生产系统与技术研究所）；Department of Innovative Technologies (DTI)（创新技术系）；University of Applied Science and Arts of Southern Switzerland (SUPSI)（瑞士南部应用科学与艺术大学）；Istituto Dalle Molle di studi sull’intelligenza artificiale (IDSIA)（达莫尔智能研究 institute）；Department of Mechanical Engineering（机械工程系）；Politecnico di Milano (PoliMi)（米兰理工学院）；Faculty of Informatics（信息学院）；Università della Svizzera Italiana (USI)（瑞士意大利大学）
Generalizing Robot Trajectories from Single-Context Human Demonstrations: A Probabilistic Approach
Robotics Research Centre, School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore（南洋理工大学机器人研究中心，机械与航空航天工程学院）；Delta-NTU Corporate Lab, Singapore（Delta-NTU企业实验室）
In many settings (e.g., robotics) demonstrations provide a natural way to specify tasks; however, most methods for learning from demonstrations either do not provide guarantees that the artifacts learned for the tasks, such as rewards or policies, can be safely composed and/or do not explicitly capture history dependencies. Motivated by this deficit, recent works have proposed learning Boolean task specifications, a class of Boolean non-Markovian rewards which admit well-defined composition and explicitly handle historical dependencies. This work continues this line of research by adapting max…
Bogazici University（海峡大学）；Ozyegin University（厄兹耶因大学）；University of Luebeck（吕贝克大学）
To equip robots with dexterous skills, an effective approach is to first transfer the desired skill via Learning from Demonstration (LfD), then let the robot improve it by self-exploration via Reinforcement Learning (RL). In this paper, we propose a novel LfD+RL framework, namely Adaptive Conditional Neural Movement Primitives (ACNMP), that allows efficient policy improvement in novel environments and effective skill transfer between different agents. This is achieved through exploiting the latent representation learned by the underlying Conditional Neural Process (CNP) model, and simultaneous…
Enhanced Robotic Navigation in Deformable Environments using Learning from Demonstration and Dynamic Modulation
Munich Institute of Robotics and Machine Intelligence (MIRMI), Technical University of Munich (TUM), Germany（慕尼黑机器人与机器智能研究所（MIRMI），慕尼黑技术大学（TUM））；Mohamed Bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE（穆罕默德·本·扎耶德人工智能大学）
Artificial Intelligence Research Institute FRC CSC RAS（俄罗斯科学院计算机科学与控制研究中心人工智能研究所）；Moscow Institute of Physics and Technology（莫斯科物理技术学院）；Higher School of Economics（高等经济学院）；Moscow Aviation Institute（莫斯科航空学院）
Currently, deep reinforcement learning (RL) shows impressive results in complex gaming and robotic environments. Often these results are achieved at the expense of huge computational costs and require an incredible number of episodes of interaction between the agent and the environment. There are two main approaches to improving the sample efficiency of reinforcement learning methods - using hierarchical methods and expert demonstrations. In this paper, we propose a combination of these approaches that allow the agent to use low-quality demonstrations in complex vision-based environments with …
Dynamic Movement Primitives (DMPs) is a framework for learning a point-to-point trajectory from a demonstration. Despite being widely used, DMPs still present some shortcomings that may limit their usage in real robotic applications. Firstly, at the state of the art, mainly Gaussian basis functions have been used to perform function approximation. Secondly, the adaptation of the trajectory generated by the DMP heavily depends on the choice of hyperparameters and the new desired goal position. Lastly, DMPs are a framework for `one-shot learning', meaning that they are constrained to learn from …
Interaction-Stiffness-Guided Basis Allocation in Dynamic Movement Primitives for Efficient Skill Transfer
Ningbo Institute of Materials Technology and Engineering, Chinese Academy of Sciences（中国科学院宁波材料技术与工程研究所）；University of Chinese Academy of Sciences（中国科学院大学）；The Hong Kong Polytechnic University（香港理工大学）
Hyper-Decision Transformer for Efficient Online Policy Adaptation
Carnegie Mellon University（卡内基梅隆大学）；University of Montreal（蒙特利尔大学）；Mila（米拉人工智能研究所）；MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）；UMass Amherst（马萨诸塞大学阿默斯特分校）
Towards personalized human AI interaction - adapting the behavior of AI agents using neural signatures of subjective interest
Department of Biomedical Engineering, Columbia University（生物医学工程系，哥伦比亚大学）；Data Science Institute, Columbia University（数据科学研究所，哥伦比亚大学）；National Institutes of Health（国家卫生研究院）
Politecnico di Milano（米兰理工大学）；Edge Hill University（边山大学）；Lincoln Institute for Agri-food Technology, University of Lincoln（林肯大学林肯农业食品技术研究所）
This paper presents a novel probabilistic approach to deep robot learning from demonstrations (LfD). Deep movement primitives (DMPs) are deterministic LfD model that maps visual information directly into a robot trajectory. This paper extends DMPs and presents a deep probabilistic model that maps the visual information into a distribution of effective robot trajectories. The architecture that leads to the highest level of trajectory accuracy is presented and compared with the existing methods. Moreover, this paper introduces a novel training method for learning domain-specific latent features.…
School of Computing, University of Leeds（利兹大学计算机学院）
As a user-friendly and straightforward solution for robot trajectory generation, imitation learning has been viewed as a vital direction in the context of robot skill learning. In contrast to unconstrained imitation learning which ignores possible internal and external constraints arising from environments and robot kinematics/dynamics, recent works on constrained imitation learning allow for transferring human skills to unstructured scenarios, further enlarging the application domain of imitation learning. While various constraints have been studied, e.g., joint limits, obstacle avoidance and…
IMT Atlantique（大西洋国立高等工业技术学院）；ENSTA Paris（巴黎国立高等先进技术学院）；Institut Polytechnique de Paris（巴黎理工学院）；Inria（法国国家信息与自动化研究所）
In open-ended continuous environments, robots need to learn multiple parameterised control tasks in hierarchical reinforcement learning. We hypothesise that the most complex tasks can be learned more easily by transferring knowledge from simpler tasks, and faster by adapting the complexity of the actions to the task. We propose a task-oriented representation of complex actions, called procedures, to learn online task relationships and unbounded sequences of action primitives to control the different observables of the environment. Combining both goal-babbling with imitation learning, and activ…
Mixed Reality (MR) has recently shown great success as an intuitive interface for enabling end-users to teach robots. Related works have used MR interfaces to communicate robot intents and beliefs to a co-located human, as well as developed algorithms for taking multi-modal human input and learning complex motor behaviors. Even with these successes, enabling end-users to teach robots complex motor tasks still poses a challenge because end-user communication is highly task dependent and world knowledge is highly varied. We propose a learning framework where end-users teach robots a) motion demo…
MINES ParisTech（巴黎高等矿业学院）；Honda Research Institute Europe（本田欧洲研究所）；Idiap Research Institute（Idiap研究所）
This study proposes a novel imitation learning approach for the stochastic generation of human-like rhythmic wave gestures and their modulation for effective non-verbal communication through a probabilistic formulation using joint angle data from human demonstrations. This is achieved by learning and modulating the overall expression characteristics of the gesture (e.g., arm posture, waving frequency and amplitude) in the frequency domain. The method was evaluated on simulated robot experiments involving a robot with a manipulator of 6 degrees of freedom. The results show that the method provi…
DROM: A Language-Guided Diffusion Framework for Multi-Skill Robotic Manipulation
University of Applied Science and Arts of Southern Switzerland (SUPSI)（瑞士南部应用科学与艺术大学）；Institute of Systems and Technologies for Sustainable Production (ISTePS)（可持续生产系统与技术研究所）；Istituto Dalle Molle di studi sull’intelligenza artificiale (IDSIA)（达勒莫尔人工智能研究所）
Mechanistic Finetuning of Vision-Language-Action Models via Few-Shot Demonstrations
Carnegie Mellon University（卡内基梅隆大学）；University of Southern California（南加州大学）；University of California, Berkeley（加州大学伯克利分校）；MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）
Motion Priors Reimagined: Adapting Flat-Terrain Skills for Complex Quadruped Mobility
Department of Mechanical Engineering, EPFL（瑞士联邦理工学院机械工程系）；ETH AI Center, ETH Zurich（苏黎世联邦理工学院人工智能中心）；Robotic Systems Lab, ETH Zurich（苏黎世联邦理工学院机器人系统实验室）
