Capability-based Frameworks for Industrial Robot Skills: a Survey
Siemens Technology（西门子技术）；Institute of Robotics and Mechatronics, German Aerospace Center (DLR)（德国航空航天中心机器人与机电一体化研究所）；Chair of Human-centered Assistive Robotics, Technical University of Munich (TUM)（慕尼黑工业大学以人为中心的辅助机器人学教席）；Autonomous Systems, Technische Universität Wien (TU Wien)（维也纳工业大学自主系统研究所）
Matteo Pantano, Thomas Eiband, Dongheui Lee
发表信息：
2022 IEEE 18th International Conference on Automation Science and Engineering (CASE)
研究界在描述机器人的能力时，对skill、action、atomic unit等词汇感到困惑。然而，为了使在工业场景中集成能力成为可能，必须对这些描述进行标准化。本文采用结构化审查方法，识别机器人技能框架研究界的共性与差异。通过该方法，分析了210篇论文，获得了三个主要结果。首先，绝大多数作者同意基于task、skill和primitive的分类法。其次，研究最多的机器人能力是pick and place。第三，面向工业的应用更关注具有固定参数的简单机器人能力，同时确保安全方面。因此，本文强调未来的工作应使用基于task、skill和primitive的分类法，以与现有文献保持一致。此外，在确保安全的同时，工业领域还需要对参数化机器人能力进行进一步研究。
英文摘要：
The research community is puzzled with words like skill, action, atomic unit and others when describing robots' capabilities. However, for giving the possibility to integrate capabilities in industrial scenarios, a standardization of these descriptions is necessary. This work uses a structured review approach to identify commonalities and differences in the research community of robots' skill frameworks. Through this method, 210 papers were analyzed and three main results were obtained. First, the vast majority of authors agree on a taxonomy based on task, skill and primitive. Second, the most investigated robots' capabilities are pick and place. Third, industrial oriented applications focus more on simple robots' capabilities with fixed parameters while ensuring safety aspects. Therefore, this work emphasizes that a taxonomy based on task, skill and primitives should be used by future works to align with existing literature. Moreover, further research is needed in the industrial domain for parametric robots' capabilities while ensuring safety.
Bosch Center for Artificial Intelligence（博世人工智能中心）
Enabling robots to quickly learn manipulation skills is an important, yet challenging problem. Such manipulation skills should be flexible, e.g., be able adapt to the current workspace configuration. Furthermore, to accomplish complex manipulation tasks, robots should be able to sequence several skills and adapt them to changing situations. In this work, we propose a rapid robot skill-sequencing algorithm, where the skills are encoded by object-centric hidden semi-Markov models. The learned skill models can encode multimodal (temporal and spatial) trajectory distributions. This approach signif…
Lund University（隆德大学）；University of Patras（帕特雷大学）；Stanford University（斯坦福大学）
Robot skills systems are meant to reduce robot setup time for new manufacturing tasks. Yet, for dexterous, contact-rich tasks, it is often difficult to find the right skill parameters. One strategy is to learn these parameters by allowing the robot system to learn directly on the task. For a learning problem, a robot operator can typically specify the type and range of values of the parameters. Nevertheless, given their prior experience, robot operators should be able to help the learning process further by providing educated guesses about where in the parameter space potential optimal solutio…
Technical University of Denmark（丹麦技术大学）；RiACT ApS（RiACT ApS公司）；Lund University（隆德大学）
The application of robotic solutions to small-batch production is challenging: economical constraints tend to dramatically limit the time for setting up new batches. Organizing robot tasks into modular software components, called skills, and allowing the assignment of multiple concurrent tasks to a single robot is potentially game-changing. However, due to cycle time constraints, it may be necessary for a skill to take over without waiting on another to terminate, and the available literature lacks a systematic approach in this case. In the present article, we fill the gap by (a) establishing …
PRISM: Precision and contact-rich Real-world Industrial Skill dataset with Multimodal sensing
State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University（北京大学智能科学与技术学院通用人工智能国家重点实验室）；Delta Intelligence（三角洲智能公司）；PKU-Wuhan Institute for Artificial Intelligence（北京大学武汉人工智能研究院）；Hubei Humanoid Robot Innovation Center Co., Ltd.（湖北人形机器人创新中心有限公司）；China Academy of Information and Communications Technology（中国信息通信研究院）
College of Engineering, Peking University（北京大学工学院）；Bosch Center for Artificial Intelligence (BCAI)（博世人工智能中心）
Learning from demonstration (LfD) provides a fast, intuitive and efficient framework to program robot skills, which has gained growing interest both in research and industrial applications. Most complex manipulation tasks are long-term and involve a set of skill primitives. Thus it is crucial to have a reliable coordination scheme that selects the correct sequence of skill primitive and the correct parameters for each skill, under various scenarios. Instead of relying on a precise simulator, this work proposes a human-in-the-loop coordination framework for LfD skills that: builds parameterized…
Lund University（隆德大学）；University of Patras（帕特雷大学）；Stanford University（斯坦福大学）
In modern industrial settings with small batch sizes it should be easy to set up a robot system for a new task. Strategies exist, e.g. the use of skills, but when it comes to handling forces and torques, these systems often fall short. We introduce an approach that provides a combination of task-level planning with targeted learning of scenario-specific parameters for skill-based systems. We propose the following pipeline: (1) the user provides a task goal in the planning language PDDL, (2) a plan (i.e., a sequence of skills) is generated and the learnable parameters of the skills are automati…
Learning-Based Strategy for Composite Robot Assembly Skill Adaptation
Chair of Machine Tools and Control System, RPTU University Kaiserslautern-Landau（机床与控制系统系，凯撒斯劳滕-兰道大学）；Innovative Factory Systems, German Institute of Artificial Intelligence（创新工厂系统，德国人工智能研究中心）
Teams of heterogeneous autonomous robots become increasingly important due to their facilitation of various complex tasks. For such heterogeneous robots, there is currently no consistent way of describing the functions that each robot provides. In the field of manufacturing, capability modeling is considered a promising approach to semantically model functions provided by different machines. This contribution investigates how to apply and extend capability models from manufacturing to the field of autonomous robots and presents an approach for such a capability model.
Lund University（隆德大学）；University of Patras（帕特雷大学）；Stanford University（斯坦福大学）
One of today's goals for industrial robot systems is to allow fast and easy provisioning for new tasks. Skill-based systems that use planning and knowledge representation have long been one possible answer to this. However, especially with contact-rich robot tasks that need careful parameter settings, such reasoning techniques can fall short if the required knowledge not adequately modeled. We show an approach that provides a combination of task-level planning and reasoning with targeted learning of skill parameters for a task at hand. Starting from a task goal formulated in PDDL, the learnabl…
Embodied Intelligence for Flexible Manufacturing: A Survey
National University of Defense Technology（中国人民解放军国防科技大学）；Wuhan University（武汉大学）；Shenzhen University（深圳大学）；Hunan University（湖南大学）；Shenyang Institute of Automation Chinese Academy of Sciences（中国科学院沈阳自动化研究所）
In this paper we introduce a novel framework for expressing and learning force-sensitive robot manipulation skills. It is based on a formalism that extends our previous work on adaptive impedance control with meta parameter learning and compatible skill specifications. This way the system is also able to make use of abstract expert knowledge by incorporating process descriptions and quality evaluation metrics. We evaluate various state-of-the-art schemes for the meta parameter learning and experimentally compare selected ones. Our results clearly indicate that the combination of our adaptive i…
Bosch Center for Artificial Intelligence（博世人工智能中心）
Complex manipulation tasks can contain various execution branches of primitive skills in sequence or in parallel under different scenarios. Manual specifications of such branching conditions and associated skill parameters are not only error-prone due to corner cases but also quickly untraceable given a large number of objects and skills. On the other hand, learning from demonstration has increasingly shown to be an intuitive and effective way to program such skills for industrial robots. Parameterized skill representations allow generalization over new scenarios, which however makes the plann…
A*STAR Institute for Infocomm Research (I2R)（新加坡科技研究局资讯通信研究院）；A*STAR Human-centric AI (CHEEM) Programme, Institute of High Performance Computing（新加坡科技研究局高性能计算研究院人类中心人工智能项目）；A*STAR Advanced Remanufacturing and Technology Centre (ARTC)（新加坡科技研究局先进再制造与技术中心）
In this paper we present a framework to learn skills from human demonstrations in the form of geometric nullspaces, which can be executed using a robot. We collect data of human demonstrations, fit geometric nullspaces to them, and also infer their corresponding geometric constraint models. These geometric constraints provide a powerful mathematical model as well as an intuitive representation of the skill in terms of the involved objects. To execute the skill using a robot, we combine this geometric skill description with the robot's kinematics and other environmental constraints, from which …
Generalizable Skill Learning for Construction Robots with Crowdsourced Natural Language Instructions, Composable Skills Standardization, and Large Language Model
Virginia Tech（弗吉尼亚理工大学）；University of Michigan（密歇根大学）
University of Stuttgart（斯图加特大学）；Bosch Center for Artificial Intelligence (BCAI)（博世人工智能中心）；Bosch Corporate Research（博世企业研究）
Learning from Demonstration (LfD) provides an intuitive and fast approach to program robotic manipulators. Task parameterized representations allow easy adaptation to new scenes and online observations. However, this approach has been limited to pose-only demonstrations and thus only skills with spatial and temporal features. In this work, we extend the LfD framework to address forceful manipulation skills, which are of great importance for industrial processes such as assembly. For such skills, multi-modal demonstrations including robot end-effector poses, force and torque readings, and opera…
University of Twente（特文特大学）；Sapienza University（罗马第一大学）；Bosch Center for AI（博世人工智能中心）
For performing robotic manipulation tasks, the core problem is determining suitable trajectories that fulfill the task requirements. Various approaches to compute such trajectories exist, being learning and optimization the main driving techniques. Our work builds on the learning-from-demonstration (LfD) paradigm, where an expert demonstrates motions, and the robot learns to imitate them. However, expert demonstrations are not sufficient to capture all sorts of task specifications, such as the timing to grasp an object. In this paper, we propose a new method that considers formal task specific…
In recent years, industrial robots have been installed in various industries to handle advanced manufacturing and high precision tasks. However, further integration of industrial robots is hampered by their limited flexibility, adaptability and decision making skills compared to human operators. Assembly tasks are especially challenging for robots since they are contact-rich and sensitive to even small uncertainties. While reinforcement learning (RL) offers a promising framework to learn contact-rich control policies from scratch, its applicability to high-dimensional continuous state-action s…
FZI Research Center for Information Technology（FZI信息科技研究中心）；IAR Institute for Anthropomatics and Robotics, HIS Humanoids and Intelligence Systems Lab, KIT Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院（KIT）IAR人形机器人与机器人学研究所、HIS人形与智能系统实验室）
Robotic automation is a key driver for the advancement of technology. The skills of human workers, however, are difficult to program and seem currently unmatched by technical systems. In this work we present a data-driven approach to extract and learn robot-independent contact skills from human demonstrations in simulation environments, using a Long Short Term Memory (LSTM) network. Our model learns to generate error-correcting sequences of forces and torques in task space from object-relative motion, which industrial robots carry out through a Cartesian force control scheme on the real setup.…
IROSA: Interactive Robot Skill Adaptation using Natural Language
German Aerospace Center (DLR), Institute of Robotics and Mechatronics (RMC)（德国航空航天中心（DLR）机器人与机电研究所）；School of Computation, Information and Technology (CIT), Technical University of Munich (TUM)（计算、信息与技术学院（CIT），慕尼黑技术大学）
Hunan University（湖南大学）；New Jersey Institute of Technology（新泽西理工学院）
Object manipulation is a basic element in everyday human lives. Robotic manipulation has progressed from maneuvering single-rigid-body objects with firm grasping to maneuvering soft objects and handling contact-rich actions. Meanwhile, technologies such as robot learning from demonstration have enabled humans to intuitively train robots. This paper discusses a new level of robotic learning-based manipulation. In contrast to the single form of learning from demonstration, we propose a multiform learning approach that integrates additional forms of skill acquisition, including adaptive learning …
MOMO: A framework for seamless physical, verbal, and graphical robot skill learning and adaptation
German Aerospace Center (DLR), Institute of Robotics and Mechatronics (RMC)（德国航空航天中心（DLR）机器人与机电研究所）；School of Computation, Information and Technology (CIT), Technical University of Munich (TUM)（计算、信息与技术学院（CIT），慕尼黑技术大学）
Institute for Artificial Intelligence (IAI), University of Bremen（不来梅大学人工智能研究所（IAI））；ArtiMinds Robotics（ArtiMinds机器人公司）
Challenging manipulation tasks can be solved effectively by combining individual robot skills, which must be parameterized for the concrete physical environment and task at hand. This is time-consuming and difficult for human programmers, particularly for force-controlled skills. To this end, we present Shadow Program Inversion (SPI), a novel approach to infer optimal skill parameters directly from data. SPI leverages unsupervised learning to train an auxiliary differentiable program representation ("shadow program") and realizes parameter inference via gradient-based model inversion. Our meth…
