Robotic force-based compliance control is a preferred approach to achieve high-precision assembly tasks. When the geometric features of assembly objects are asymmetric or irregular, reinforcement learning (RL) agents are gradually incorporated into the compliance controller to adapt to complex force-pose mapping which is hard to model analytically. Since force-pose mapping is strongly dependent on geometric features, a compliance controller is only optimal for current geometric features. To reduce the learning cost of assembly objects with different geometric features, this paper is devoted to answering how to reconfigure existing controllers for new assembly objects with different geometric features. In this paper, model-based parameters are first reconfigured based on the proposed Equivalent Theory of Compliance Law (ETCL). Then the RL agent is transferred based on the proposed Weighted Dimensional Policy Distillation (WDPD) method. The experiment results demonstrate that the control reconfiguration method costs less time and achieves better control performance, which confirms the validity of proposed methods.
Ruhr University Bochum（鲁尔大学波鸿分校）；University of Bergamo（贝加莫大学）；Chalmers University of Technology（查尔姆斯理工大学）；University of Gothenburg（哥德堡大学）；Radboud University Nijmegen（奈梅亨拉德堡德大学）；Gran Sasso Science Institute (GSSI)（格兰萨索科学研究所）；Bergen University（卑尔根大学）
University of Bourgogne Franche-Comté（勃艮第-弗朗什-孔泰大学）；FEMTO-ST Institute（FEMTO-ST研究所）；CNRS（法国国家科学研究中心）；Warsaw University of Technology（华沙理工大学）；Institute of Fundamental Technological Research Polish Academy of Sciences（波兰科学院基础技术研究所）；University of Luxembourg（卢森堡大学）
We present a distributed framework for predicting whether a planned reconfiguration step of a modular robot will mechanically overload the structure, causing it to break or lose stability under its own weight. The algorithm is executed by the modular robot itself and based on a distributed iterative solution of mechanical equilibrium equations derived from a simplified model of the robot. The model treats inter-modular connections as beams and assumes no-sliding contact between the modules and the ground. We also provide a procedure for simplified instability detection. The algorithm is verifi…
Traditional control methods of robotic peg-in-hole assembly rely on complex contact state analysis. Reinforcement learning (RL) is gradually becoming a preferred method of controlling robotic peg-in-hole assembly tasks. However, the training process of RL is quite time-consuming because RL methods are always globally connected, which means all state components are assumed to be the input of policies for all action components, thus increasing action space and state space to be explored. In this paper, we first define continuous space serialized Shapley value (CS3) and construct a connection gra…
In this paper, our focus is on certain applications for mobile robotic networks, where reconfiguration is driven by factors intrinsic to the network rather than changes in the external environment. In particular, we study a version of the coverage problem useful for surveillance applications, where the objective is to position the robots in order to minimize the average distance from a random point in a given environment to the closest robot. This problem has been well-studied for omni-directional robots and it is shown that optimal configuration for the network is a centroidal Voronoi configu…
In this work, we propose a framework called Auto-Assembly for automated robotic assembly from design files and demonstrate a practical implementation on modular parts joined by fastening using a robotic cell consisting of two robots. We show the flexibility of the approach by testing it on different input designs. Auto-Assembly consists of several parts: design analysis, assembly sequence generation, bill-of-process (BOP) generation, conversion of the BOP to control code, path planning, simulation, and execution of the control code to assemble parts in the physical environment.
Exploring environment exploitation for self-reconfiguration in modular robotics
Department of Applied Mathematics University of Washington（应用数学系 华盛顿大学）；Department of Mechanical Engineering and Applied Mechanics University of Pennsylvania（机械工程与应用力学系 费城大学）；Department of Mechanical Engineering University of Nevada, Las Vegas（机械工程系 内华达大学拉斯维加斯分校）
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Carnegie Mellon University（卡内基梅隆大学）
Significant progress in robotics reveals new opportunities to advance manufacturing. Next-generation industrial automation will require both integration of distinct robotic technologies and their application to challenging industrial environments. This paper presents lessons from a collaborative assembly project between three academic research groups and an industry partner. The goal of the project is to develop a flexible, safe, and productive manufacturing cell for sub-centimeter precision assembly. Solving this problem in a high-mix, low-volume production line motivates multiple research th…
We introduce a robotic assembly system that streamlines the design-to-make workflow for going from a CAD model of a product assembly to a fully programmed and adaptive assembly process. Our system captures (in the CAD tool) the intent of the assembly process for a specific robotic workcell and generates a recipe of task-level instructions. By integrating visual sensing with deep-learned perception models, the robots infer the necessary actions to assemble the design from the generated recipe. The perception models are trained directly from simulation, allowing the system to identify various pa…
Vision-based robotic assembly is a crucial yet challenging task as the interaction with multiple objects requires high levels of precision. In this paper, we propose an integrated 6D robotic system to perceive, grasp, manipulate and assemble blocks with tight tolerances. Aiming to provide an off-the-shelf RGB-only solution, our system is built upon a monocular 6D object pose estimation network trained solely with synthetic images leveraging physically-based rendering. Subsequently, pose-guided 6D transformation along with collision-free assembly is proposed to construct any designed structure …
Fine robotic assembly, in which the parts to be assembled are small and fragile and lie in an unstructured environment, is still out of reach of today's industrial robots. The main difficulties arise in the precise localization of the parts in an unstructured environment and the control of contact interactions. Our contribution in this paper is twofold. First, we propose a taxonomy of the manipulation primitives that are specifically involved in fine assembly. Such a taxonomy is crucial for designing a scalable robotic system (both hardware and software) given the complexity of real-world asse…
Active Force Control (AFC) is an important scheme for tackling high-precision robotic assembly. Classical force controllers are highly surface-dependent: the controller must be carefully tuned for each type of surface in contact, in order to avoid instabilities and to achieve a reasonable performance level. Here, we build upon the recently-developed Convex Controller Synthesis (CCS) to enable high-precision assembly across a wide range of surface stiffnesses without any surface-dependent tuning. Specifically, we demonstrate peg-in-hole assembly with 100 micron clearance, initial position uncer…
NVIDIA Corporation（英伟达公司）；University of Washington（华盛顿大学）
Robotic assembly is one of the oldest and most challenging applications of robotics. In other areas of robotics, such as perception and grasping, simulation has rapidly accelerated research progress, particularly when combined with modern deep learning. However, accurately, efficiently, and robustly simulating the range of contact-rich interactions in assembly remains a longstanding challenge. In this work, we present Factory, a set of physics simulation methods and robot learning tools for such applications. We achieve real-time or faster simulation of a wide range of contact-rich scenes, inc…
Tsinghua University（清华大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
In this paper, we present an overview of robotic peg-in-hole assembly and analyze two main strategies: contact model-based and contact model-free strategies. More specifically, we first introduce the contact model control approaches, including contact state recognition and compliant control two steps. Additionally, we focus on a comprehensive analysis of the whole robotic assembly system. Second, without the contact state recognition process, we decompose the contact model-free learning algorithms into two main subfields: learning from demonstrations and learning from environments (mainly base…
Osaka University（大阪大学）；University of California, Santa Cruz（加州大学圣克鲁兹分校）
This paper develops a robotic manipulation planner for human-robot collaborative assembly. Unlike previous methods which study an independent and fully AI-equipped autonomous system, this paper explores the subtask distribution between a robot and a human and studies a human-in-the-loop robotic system for collaborative assembly. The system distributes the subtasks of an assembly to robots and humans by exploiting their advantages and avoiding their disadvantages. The robot in the system will work on pick-and-place tasks and provide workpieces to humans. The human collaborator will work on fine…
Despite their contributions to the financial efficiency and environmental sustainability of industrial processes, robotic assembly and disassembly have been understudied in the existing literature. This is in contradiction to their importance in realizing the Fourth Industrial Revolution. More specifically, although most of the literature has extensively discussed how to optimally assemble or disassemble given products, the role of other factors has been overlooked. For example, the types of robots involved in implementing the sequence plans, which should ideally be taken into account througho…
Georgia Institute of Technology（佐治亚理工学院）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）
This paper presents the foundation of a modular robotic system comprised of several novel modules in the shape of a tetrahedron. Four single-propeller submodules are assembled to create the Tetracopter, a tetrahedron-shaped quad-rotorcraft used as the elementary module of a modular flying system. This modular flying system is built by assembling the different elementary modules in a fractal shape. The fractal tetrahedron structure of the modular flying assembly grants the vehicle more rigidity than a conventional two-dimensional modular robotic flight system while maintaining the relative effi…
Speech to Reality: On-Demand Production using Natural Language, 3D Generative AI, and Discrete Robotic Assembly
Computer Science and Artificial Intelligence Lab（计算机科学与人工智能实验室）；Massachusetts Institute of Technology（麻省理工学院）；Center for Bits and Atoms（比特与原子中心）；Department of Mechanical Engineering（机械工程系）
University of Maryland, College Park（马里兰大学帕克分校）；King Abdullah University of Science & Technology（阿卜杜拉国王科技大学）
We present the design, characterization, and experimental results for a new modular robotic system for programmable self-assembly. The proposed system uses the Hybrid Cube Model (HCM), which integrates classical features from both deterministic and stochastic self-organization models. Thus, for instance, the modules are passive as far as their locomotion is concerned (stochastic), and yet they possess an active undocking routine (deterministic). The robots are constructed entirely from readily accessible components and unlike many existing robots, their excitation is not fluid mediated. Instea…
Mitsubishi Electric Research Laboratories（三菱电机研究实验室）
Insertion operations are a critical element of most robotic assembly operation, and peg-in-hole (PiH) insertion is one of the most widely studied tasks in the industrial and academic manipulation communities. PiH insertion is in fact an entire class of problems, where the complexity of the problem can depend on the type of misalignment and contact formation during an insertion attempt. In this paper, we present the design and analysis of adaptive compliance controllers which can be used in insertion-type assembly tasks, including learning-based compliance controllers which can be used for inse…
Backward Layout Search for Sequence-Constrained Robotic Assembly
College of Computer and Information Sciences, Fujian Agriculture and Forestry University（福建农林大学计算机与信息学院）；School of Mechatronic Engineering and Automation, Shanghai University（上海大学机电工程与自动化学院）；Department of System Innovation, Graduate School of Engineering Science, Osaka University（大阪大学工程科学研究生院系统创新系）
Autonomous Integration and Improvement of Robotic Assembly using Skill Graph Representations
Department of Electrical and Computer Engineering, Carnegie Mellon University（卡内基梅隆大学电气与计算机工程系）；Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）
Graduate School of Engineering Science, Osaka University（大阪大学工程科学研究生院）；Chinese University of Hong Kong（香港中文大学）
Existing methods for predicting robotic snap joint assembly cannot predict failures before their occurrence. To address this limitation, this paper proposes a method for predicting error states before the occurence of error, thereby enabling timely recovery. Robotic snap joint assembly requires precise positioning; therefore, even a slight offset between parts can lead to assembly failure. To correctly predict error states, we apply functional principal component analysis (fPCA) to 6D force/torque profiles that are terminated before the occurence of an error. The error state is identified by a…
REASSEMBLE: A Multimodal Dataset for Contact-rich Robotic Assembly and Disassembly
Autonomous Systems Lab, Institute of Computer Technology, TU Wien（自主系统实验室，计算机技术学院，维也纳技术大学）；Institute of Robotics and Mechatronics (DLR), German Aerospace Center（机器人与机电系统研究所（DLR），德国航空航天中心）
AssemLM: A Spatial Reasoning Multimodal Large Language Model for Robotic Assembly
Fudan University（复旦大学）；Institute of Artificial Intelligence (TeleAI), China Telecom（人工智能研究所（TeleAI），中国电信）；Tianjin University（天津大学）；Northwestern Polytechnical University（西北工业大学）；Tsinghua University（清华大学）；City University of Hong Kong（香港城市大学）
Osaka University（大阪大学）；National Institute of Advanced Industrial Science and Technology（国立先进工业科学技术研究院）
Robotic assembly tasks involve complex and low-clearance insertion trajectories with varying contact forces at different stages. While the nominal motion trajectory can be easily obtained from human demonstrations through kinesthetic teaching, teleoperation, simulation, among other methods, the force profile is harder to obtain especially when a real robot is unavailable. It is difficult to obtain a realistic force profile in simulation even with physics engines. Such simulated force profiles tend to be unsuitable for the actual robotic assembly due to the reality gap and uncertainty in the as…
WorkBenchMark: A LEGO-Based Assembly Benchmark with an Assembly-by-Disassembly Baseline for the Smart Manufacturing League
Chair of Machine Learning and Reasoning (i6), RWTH Aachen University（亚琛工业大学机器学习与推理教席（i6））；MASCOR Institute, FH Aachen University of Applied Science（亚琛应用技术大学MASCOR研究所）
While the ultimate goal is to produce a tensegrity more than 6 struts, e.g. a 15-bar tensegrity, past experience has demonstrated that we must first develop an innovative system that will facilitate the assembly of a general n-bar tensegrity. To be successful, we believe the development of the new assembly methodology must encompass not only the design of the clamping system but also the design of the tensegrity itself, including the struts, the springs and the spring-to-strut connectors. We therefore propose to develop the 15-bar in two phases: Phase I will be the development of an innovative…
Seoul National University（首尔国立大学）；Samsung Research（三星研究院）
We propose a novel fast and accurate simulation framework for contact-intensive tight-tolerance robotic assembly tasks. The key components of our framework are as follows: 1) data-driven contact point clustering with a certain variable-input network, which is explicitly trained for simulation accuracy (with real experimental data) and able to accommodate complex/non-convex object shapes; 2) contact force solving, which precisely/robustly enforces physics of contact (i.e., no penetration, Coulomb friction, maximum energy dissipation) with contact mechanics of contact nodes augmented with that o…
X, The Moonshot Factory（X公司，Moonshot工厂）；New England Robotics Validation and Experimentation (NERVE) Center, University of Massachusetts Lowell（马萨诸塞大学洛厄尔分校，新英格兰机器人验证与实验中心）
In recent years, many learning based approaches have been studied to realize robotic manipulation and assembly tasks, often including vision and force/tactile feedback. However, it remains frequently unclear what is the baseline state-of-the-art performance and what are the bottleneck problems. In this work, we evaluate some off-the-shelf (OTS) industrial solutions on a recently introduced benchmark, the National Institute of Standards and Technology (NIST) Assembly Task Boards. A set of assembly tasks are introduced and baseline methods are provided to understand their intrinsic difficulty. M…
Massachusetts Institute of Technology（麻省理工学院）；Institute for Systems and Robotics, Instituto Superior Técnico（高等技术学院系统与机器人研究所）
As access to space and robotic autonomy capabilities move forward, there is simultaneously a growing interest in deploying large, complex space structures to provide new on-orbit capabilities. New space-borne observatories, large orbital outposts, and even futuristic on-orbit manufacturing will be enabled by robotic assembly of space structures using techniques like on-orbit additive manufacturing which can provide flexibility in constructing and even repairing complex hardware. However, the dynamics underlying the robotic assembler during manipulation may operate under inertial uncertainties.…
Singapore Centre for 3D Printing (SC3DP)（新加坡3D打印中心）；School of Mechanical and Aerospace Engineering, NTU（南洋理工大学机械与航天工程学院）；Eureka Robotics（尤里卡机器人公司）
This paper explores the idea that skillful assembly is best represented as dynamic sequences of Manipulation Primitives, and that such sequences can be automatically discovered by Reinforcement Learning. Manipulation Primitives, such as "Move down until contact", "Slide along x while maintaining contact with the surface", have enough complexity to keep the search tree shallow, yet are generic enough to generalize across a wide range of assembly tasks. Moreover, the additional "semantics" of the Manipulation Primitives make them more robust in sim2real and against model/environment variations a…
Most of industrial robotic assembly tasks today require fixed initial conditions for successful assembly. These constraints induce high production costs and low adaptability to new tasks. In this work we aim towards flexible and adaptable robotic assembly by using 3D CAD models for all parts to be assembled. We focus on a generic assembly task - the Siemens Innovation Challenge - in which a robot needs to assemble a gear-like mechanism with high precision into an operating system. To obtain the millimeter-accuracy required for this task and industrial settings alike, we use a depth camera moun…
FZI Research Center for Information Technology（FZI信息技术研究中心）
The programming of robotic assembly tasks is a key component in manufacturing and automation. Force-sensitive assembly, however, often requires reactive strategies to handle slight changes in positioning and unforeseen part jamming. Learning such strategies from human performance is a promising approach, but faces two common challenges: the handling of low part clearances which is difficult to capture from demonstrations and learning intuitive strategies offline without access to the real hardware. We address these two challenges by learning probabilistic force strategies from data that are ea…
Technische Universität München（慕尼黑工业大学）；Agile Robots AG（敏捷机器人公司）；Universität Hamburg（汉堡大学）
Recently, deep reinforcement learning (RL) has shown some impressive successes in robotic manipulation applications. However, training robots in the real world is nontrivial owing to sample efficiency and safety concerns. Sim-to-real transfer is proposed to address the aforementioned concerns but introduces a new issue called the reality gap. In this work, we introduce a sim-to-real learning framework for vision-based assembly tasks and perform training in a simulated environment by employing inputs from a single camera to address the aforementioned issues. We present a domain adaptation metho…
