Using Reinforcement Learning (RL) to learn new robotic tasks from scratch is often inefficient. Leveraging prior knowledge has the potential to significantly enhance learning efficiency, which, however, raises two critical challenges: how to determine the relevancy of existing knowledge and how to adaptively integrate them into learning a new task. In this paper, we propose Context-aware Adaptation for Robot Learning (CARoL), a novel framework to efficiently learn a similar but distinct new task from prior knowledge. CARoL incorporates context awareness by analyzing state transitions in system dynamics to identify similarities between the new task and prior knowledge. It then utilizes these identified similarities to prioritize and adapt specific knowledge pieces for the new task. Additionally, CARoL has a broad applicability spanning policy-based, value-based, and actor-critic RL algorithms. We validate the efficiency and generalizability of CARoL on both simulated robotic platforms and physical ground vehicles. The simulations include CarRacing and LunarLander environments, where CARoL demonstrates faster convergence and higher rewards when learning policies for new tasks. In real-world experiments, we show that CARoL enables a ground vehicle to quickly and efficiently adapt policies learned in simulation to smoothly traverse real-world off-road terrain.
Istituto Italiano di Tecnologia（意大利技术研究院）；University of Genoa（热那亚大学）；Politecnico di Milano（米兰理工大学）
Despite cobots have high potential in bringing several benefits in the manufacturing and logistic processes, but their rapid (re-)deployment in changing environments is still limited. To enable fast adaptation to new product demands and to boost the fitness of the human workers to the allocated tasks, we propose a novel method that optimizes assembly strategies and distributes the effort among the workers in human-robot cooperative tasks. The cooperation model exploits AND/OR Graphs that we adapted to solve also the role allocation problem. The allocation algorithm considers quantitative measu…
Robotic grasping is one of the most fundamental robotic manipulation tasks and has been actively studied. However, how to quickly teach a robot to grasp a novel target object in clutter remains challenging. This paper attempts to tackle the challenge by leveraging object attributes that facilitate recognition, grasping, and quick adaptation. In this work, we introduce an end-to-end learning method of attribute-based robotic grasping with one-grasp adaptation capability. Our approach fuses the embeddings of a workspace image and a query text using a gated-attention mechanism and learns to predi…
University of South Carolina, USA（南卡罗来纳大学）；University of York, UK（约克大学）；Carnegie Mellon University, USA（卡内基梅隆大学）
Modern cyber-physical systems (e.g., robotics systems) are typically composed of physical and software components, the characteristics of which are likely to change over time. Assumptions about parts of the system made at design time may not hold at run time, especially when a system is deployed for long periods (e.g., over decades). Self-adaptation is designed to find reconfigurations of systems to handle such run-time inconsistencies. Planners can be used to find and enact optimal reconfigurations in such an evolving context. However, for systems that are highly configurable, such planning b…
Few-Shot Adaptation to Non-Stationary Environments via Latent Trend Embedding for Robotics
College of Information Science and Engineering, Ritsumeikan University（立命馆大学信息科学与工程学院）；Production & Technology Department, NIPPN CORPORATION（NIPPN公司生产与技术部门）；University of Osaka（大阪大学）；National Institute of Technology, Ichinoseki College（国立Ichinoseki学院）
Escaping the Diversity Trap in Robotic Manipulation via Anchor-Centric Adaptation
National University of Singapore（新加坡国立大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；Institude for Information Research, A STAR（A*STAR信息研究所）
Task-agnostic Lifelong Robot Learning with Retrieval-based Weighted Local Adaptation
College of Design and Engineering National University of Singapore（设计与工程学院新加坡国立大学）；Jacobs School of Engineering University of California San Diego（工程学院加州大学圣地亚哥分校）；Massachusetts General Hospital Harvard University（麻省总医院哈佛大学）；Faculty of Electrical Engineering, Mathematics and Computer Science Delft University of Technology（电子工程、数学和计算机科学学院代尔夫特技术大学）；Faculty of Mechanical Engineering Delft University of Technology（机械工程学院代尔夫特技术大学）
Successful real-world deployment of legged robots would require them to adapt in real-time to unseen scenarios like changing terrains, changing payloads, wear and tear. This paper presents Rapid Motor Adaptation (RMA) algorithm to solve this problem of real-time online adaptation in quadruped robots. RMA consists of two components: a base policy and an adaptation module. The combination of these components enables the robot to adapt to novel situations in fractions of a second. RMA is trained completely in simulation without using any domain knowledge like reference trajectories or predefined …
We explore possible methods for multi-task transfer learning which seek to exploit the shared physical structure of robotics tasks. Specifically, we train policies for a base set of pre-training tasks, then experiment with adapting to new off-distribution tasks, using simple architectural approaches for re-using these policies as black-box priors. These approaches include learning an alignment of either the observation space or action space from a base to a target task to exploit rigid body structure, and methods for learning a time-domain switching policy across base tasks which solves the ta…
University of Hamburg（汉堡大学）；SoftBank Robotics Europe（软银机器人欧洲公司）
Service robots need to show appropriate social behaviour in order to be deployed in social environments such as healthcare, education, retail, etc. Some of the main capabilities that robots should have are navigation and conversational skills. If the person is impatient, the person might want a robot to navigate faster and vice versa. Linguistic features that indicate politeness can provide social cues about a person's patient and impatient behaviour. The novelty presented in this paper is to dynamically incorporate politeness in robotic dialogue systems for navigation. Understanding the polit…
Recent successes in machine learning have led to a shift in the design of autonomous systems, improving performance on existing tasks and rendering new applications possible. Data-focused approaches gain relevance across diverse, intricate applications when developing data collection and curation pipelines becomes more effective than manual behaviour design. The following work aims at increasing the efficiency of this pipeline in two principal ways: by utilising more powerful sources of informative data and by extracting additional information from existing data. In particular, we target three…
IROSA: Interactive Robot Skill Adaptation using Natural Language
German Aerospace Center (DLR), Institute of Robotics and Mechatronics (RMC)（德国航空航天中心（DLR）机器人与机电研究所）；School of Computation, Information and Technology (CIT), Technical University of Munich (TUM)（计算、信息与技术学院（CIT），慕尼黑技术大学）
The high prevalence of work-related musculoskeletal disorders (WMSDs) could be addressed by optimizing Human-Robot Collaboration (HRC) frameworks for manufacturing applications. In this context, this paper proposes two hypotheses for ergonomically effective task delegation and HRC. The first hypothesis states that it is possible to quantify ergonomically professional tasks using motion data from a reduced set of sensors. Then, the most dangerous tasks can be delegated to a collaborative robot. The second hypothesis is that by including gesture recognition and spatial adaptation, the ergonomics…
Towards Real-time Adaptation of Embodied Agent in Human-Robot Collaboration
University of Southern California Department of Electrical（南加州大学电气与计算机工程系）；University of Southern California Department of Computer Science Los Angeles CA USA（南加州大学计算机科学系洛杉矶加州美国）；University of Southern California（南加州大学）
DecompGrind: A Decomposition Framework for Robotic Grinding via Cutting-Surface Planning and Contact-Force Adaptation
Nara Institute of Science and Technology (NAIST)（奈良先端科学技术大学院大学）；Keio University（庆应义塾大学）；Kyoto University（京都大学）；Advanced Telecommunications Research Institute International (ATR)（国际电气通信基础技术研究所）
Inria（法国国家信息与自动化研究所）；CNRS（法国国家科学研究中心）；Université de Lorraine（洛林大学）；LORIA（洛林计算机科学与应用数学实验室）
Repertoire-based learning is a data-efficient adaptation approach based on a two-step process in which (1) a large and diverse set of policies is learned in simulation, and (2) a planning or learning algorithm chooses the most appropriate policies according to the current situation (e.g., a damaged robot, a new object, etc.). In this paper, we relax the assumption of previous works that a single repertoire is enough for adaptation. Instead, we generate repertoires for many different situations (e.g., with a missing leg, on different floors, etc.) and let our algorithm selects the most useful p…
Bosch Center for Artificial Intelligence（博世人工智能中心）
Flexible manufacturing processes demand robots to easily adapt to changes in the environment and interact with humans. In such dynamic scenarios, robotic tasks may be programmed through learning-from-demonstration approaches, where a nominal plan of the task is learned by the robot. However, the learned plan may need to be adapted in order to fulfill additional requirements or overcome unexpected environment changes. When the required adaptation occurs at the end-effector trajectory level, a human operator may want to intuitively show the robot the desired changes by physically interacting wit…
University of Lincoln（林肯大学）；Lincoln Institute for Agri-Food Technology（林肯农业食品技术研究所）
Robots learning a new manipulation task from a small amount of demonstrations are increasingly demanded in different workspaces. A classifier model assessing the quality of actions can predict the successful completion of a task, which can be used by intelligent agents for action-selection. This paper presents a novel classifier that learns to classify task completion only from a few demonstrations. We carry out a comprehensive comparison of different neural classifiers, e.g. fully connected-based, fully convolutional-based, sequence2sequence-based, and domain adaptation-based classification. …
Learning-based approaches to robotic manipulation are limited by the scalability of data collection and accessibility of labels. In this paper, we present a multi-task domain adaptation framework for instance grasping in cluttered scenes by utilizing simulated robot experiments. Our neural network takes monocular RGB images and the instance segmentation mask of a specified target object as inputs, and predicts the probability of successfully grasping the specified object for each candidate motor command. The proposed transfer learning framework trains a model for instance grasping in simulatio…
MOMO: A framework for seamless physical, verbal, and graphical robot skill learning and adaptation
German Aerospace Center (DLR), Institute of Robotics and Mechatronics (RMC)（德国航空航天中心（DLR）机器人与机电研究所）；School of Computation, Information and Technology (CIT), Technical University of Munich (TUM)（计算、信息与技术学院（CIT），慕尼黑技术大学）
Institute for Human and Machine Cognition (IHMC)（人机认知研究所（IHMC））；ABB Future Labs（ABB未来实验室）
Industrial robots play an increasingly important role in a growing number of fields. For example, robotics is used to increase productivity while reducing costs in various aspects of manufacturing. Since robots are often set up in production lines, the breakdown of a single robot has a negative impact on the entire process, in the worst case bringing the whole line to a halt until the issue is resolved, leading to substantial financial losses due to the unforeseen downtime. Therefore, predictive maintenance systems based on the internal signals of robots have gained attention as an essential c…
University of California Berkeley（加州大学伯克利分校）；Carnegie Mellon University（卡内基梅隆大学）
Recent advances in legged locomotion have enabled quadrupeds to walk on challenging terrains. However, bipedal robots are inherently more unstable and hence it's harder to design walking controllers for them. In this work, we leverage recent advances in rapid adaptation for locomotion control, and extend them to work on bipedal robots. Similar to existing works, we start with a base policy which produces actions while taking as input an estimated extrinsics vector from an adaptation module. This extrinsics vector contains information about the environment and enables the walking controller to …
Instrumenting and collecting annotated visual grasping datasets to train modern machine learning algorithms can be extremely time-consuming and expensive. An appealing alternative is to use off-the-shelf simulators to render synthetic data for which ground-truth annotations are generated automatically. Unfortunately, models trained purely on simulated data often fail to generalize to the real world. We study how randomized simulated environments and domain adaptation methods can be extended to train a grasping system to grasp novel objects from raw monocular RGB images. We extensively evaluate…
The quest for the efficient adaptation of multilegged robotic systems to changing conditions is expected to render new insights into robotic control and locomotion. In this paper, we study the performance frontiers of the enumerative (factorial) encoding of hexapod gaits for fast recovery to conditions of leg failures. Our computational studies using five nature-inspired gradient-free optimization heuristics have shown that it is possible to render feasible recovery gait strategies that achieve minimal deviation to desired locomotion directives with a few evaluations (trials). For instance, it…
A core challenge for an autonomous agent acting in the real world is to adapt its repertoire of skills to cope with its noisy perception and dynamics. To scale learning of skills to long-horizon tasks, robots should be able to learn and later refine their skills in a structured manner through trajectories rather than making instantaneous decisions individually at each time step. To this end, we propose the Soft Actor-Critic Gaussian Mixture Model (SAC-GMM), a novel hybrid approach that learns robot skills through a dynamical system and adapts the learned skills in their own trajectory distribu…
The population explosion of the 21st century has adversely affected the natural resources with restricted availability of cultivable land, increased average temperatures due to global warming, and carbon footprint resulting in a drastic increase in floods as well as droughts thus making food security significant anxiety for most countries. The traditional methods were no longer sufficient which paved the way for technological ascents such as a substantial rise in Artificial Intelligence (AI), Internet of Things (IoT), as well as Robotics that provides high productivity, functional efficiency, …
Stanford University（斯坦福大学）；KTH Royal Institute of Technology（瑞典皇家理工学院）
Reinforcement learning methods can achieve significant performance but require a large amount of training data collected on the same robotic platform. A policy trained with expensive data is rendered useless after making even a minor change to the robot hardware. In this paper, we address the challenging problem of adapting a policy, trained to perform a task, to a novel robotic hardware platform given only few demonstrations of robot motion trajectories on the target robot. We formulate it as a few-shot meta-learning problem where the goal is to find a meta-model that captures the common stru…
Reinforcement learning has shown a wide usage in robotics tasks, such as insertion and grasping. However, without a practical sim2real strategy, the policy trained in simulation could fail on the real task. There are also wide researches in the sim2real strategies, but most of those methods rely on heavy image rendering, domain randomization training, or tuning. In this work, we solve the insertion task using a pure visual reinforcement learning solution with minimum infrastructure requirement. We also propose a novel sim2real strategy, Real2Sim, which provides a novel and easier solution in p…
Delft University of Technology（代尔夫特理工大学）；University Libre du Bruxelles（布鲁塞尔自由大学）；Fraunhofer Institute for Manufacturing Engineering and Automation IPA（弗劳恩霍夫制造工程与自动化研究所IPA）
Self-adaptation can be used in robotics to increase system robustness and reliability. This work describes the Metacontrol method for self-adaptation in robotics. Particularly, it details how the MROS (Metacontrol for ROS Systems) framework implements and packages Metacontrol, and it demonstrate how MROS can be applied in a navigation scenario where a mobile robot navigates in a factory floor. Video: https://www.youtube.com/watch?v=ISe9aMskJuE
In human-robot teams, humans often start with an inaccurate model of the robot capabilities. As they interact with the robot, they infer the robot's capabilities and partially adapt to the robot, i.e., they might change their actions based on the observed outcomes and the robot's actions, without replicating the robot's policy. We present a game-theoretic model of human partial adaptation to the robot, where the human responds to the robot's actions by maximizing a reward function that changes stochastically over time, capturing the evolution of their expectations of the robot's capabilities. …
Volkswagen Machine Learning Research Lab（大众机器学习研究实验室）
Placing robots outside controlled conditions requires versatile movement representations that allow robots to learn new tasks and adapt them to environmental changes. The introduction of obstacles or the placement of additional robots in the workspace, the modification of the joint range due to faults or range-of-motion constraints are typical cases where the adaptation capabilities play a key role for safely performing the robot's task. Probabilistic movement primitives (ProMPs) have been proposed for representing adaptable movement skills, which are modelled as Gaussian distributions over tr…
Carnegie Mellon University（卡内基梅隆大学）；National University of Singapore（新加坡国立大学）
Shared autonomy integrates user input with robot autonomy in order to control a robot and help the user to complete a task. Our work aims to improve the performance of such a human-robot team: the robot tries to guide the human towards an effective strategy, sometimes against the human's own preference, while still retaining his trust. We achieve this through a principled human-robot mutual adaptation formalism. We integrate a bounded-memory adaptation model of the human into a partially observable stochastic decision model, which enables the robot to adapt to an adaptable human. When the huma…
Uppsala University（乌普萨拉大学）；KTH Royal Institute of Technology（KTH皇家理工学院）
In socially assistive robotics, an important research area is the development of adaptation techniques and their effect on human-robot interaction. We present a meta-learning based policy gradient method for addressing the problem of adaptation in human-robot interaction and also investigate its role as a mechanism for trust modelling. By building an escape room scenario in mixed reality with a robot, we test our hypothesis that bi-directional trust can be influenced by different adaptation algorithms. We found that our proposed model increased the perceived trustworthiness of the robot and in…
Colorado School of Mines（科罗拉多矿业学院）；Army Research Laboratory (ARL)（美国陆军研究实验室（ARL））
Ground robots require the crucial capability of traversing unstructured and unprepared terrains and avoiding obstacles to complete tasks in real-world robotics applications such as disaster response. When a robot operates in off-road field environments such as forests, the robot's actual behaviors often do not match its expected or planned behaviors, due to changes in the characteristics of terrains and the robot itself. Therefore, the capability of robot adaptation for consistent behavior generation is essential for maneuverability on unstructured off-road terrains. In order to address the ch…
University of Bristol（布里斯托大学）；Bristol Robotics Lab（布里斯托机器人实验室）；Imperial College London（帝国理工学院）；Polytechnic University of Milan（米兰理工大学）；Shanghai Jiao Tong University（上海交通大学）；Worcester Polytechnic Institute（伍斯特理工学院）
Human-robot shared control, which integrates the advantages of both humans and robots, is an effective approach to facilitate efficient surgical operation. Learning from demonstration (LfD) techniques can be used to automate some of the surgical subtasks for the construction of the shared control framework. However, a sufficient amount of data is required for the robot to learn the manoeuvres. Using a surgical simulator to collect data is a less resource-demanding approach. With sim-to-real adaptation, the manoeuvres learned from a simulator can be transferred to a physical robot. To this end,…
Integrating Flow Theory and Adaptive Robot Roles: A Conceptual Model of Dynamic Robot Role Adaptation for the Enhanced Flow Experience in Long-term Multi-person Human-Robot Interactions
Known attempts to build autonomous robots rely on complex control architectures, often implemented with the Robot Operating System platform (ROS). Runtime adaptation is needed in these systems, to cope with component failures and with contingencies arising from dynamic environments-otherwise, these affect the reliability and quality of the mission execution. Existing proposals on how to build self-adaptive systems in robotics usually require a major re-design of the control architecture and rely on complex tools unfamiliar to the robotics community. Moreover, they are hard to reuse across appl…
