Robotic assembly planning enables architects to explicitly account for the assembly process during the design phase, and enables efficient building methods that profit from the robots' different capabilities. Previous work has addressed planning of robot assembly sequences and identifying the feasibility of architectural designs. This paper extends previous work by enabling planning with large, heterogeneous teams of robots. We present a planning system which enables parallelization of complex task and motion planning problems by iteratively solving smaller subproblems. Combining optimization methods to solve for manipulation constraints with a sampling-based bi-directional space-time path planner enables us to plan cooperative multi-robot manipulation with unknown arrival-times. Thus, our solver allows for completing subproblems and tasks with differing timescales and synchronizes them effectively. We demonstrate the approach on multiple case-studies to show the robustness over long planning horizons and scalability to many objects and agents of our algorithm. Finally, we also demonstrate the execution of the computed plans on two robot arms to showcase the feasibility in the real world.
Simulation-based planning of Motion Sequences for Automated Procedure Optimization in Multi-Robot Assembly Cells
Institute for Material Handling and Logistics (IFL), Karlsruhe Institute of Technology（材料搬运与物流研究所（IFL），卡尔斯鲁厄理工学院）；Bosch Corporate Research, Robert Bosch GmbH（博世企业研究，罗伯特·博世有限公司）
Parallel Self-assembly for Modular USVs with Diverse Docking Mechanism Layouts
Shenzhen Institute of Artificial Intelligence and Robotics for Society (AIRS)（深圳市人工智能与机器人研究院）；The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen)（香港中文大学（深圳））；School of Science and Engineering (SSE), The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen)（香港中文大学（深圳）理工学院）
Multi-Robot Assembly of Deformable Linear Objects Using Multi-Modal Perception
Chair of Robotics, Artificial Intelligence and Real-time Systems, School of Computation, Information and Technology（机器人学、人工智能与实时系统教授会，计算、信息与技术学院）；Institute for Machine Tools and Industrial Management, School of Engineering and Design（机械加工与工业管理研究所，工程与设计学院）
Robotics Research Center, IIIT Hyderabad（国际信息技术研究所海得拉巴分校机器人研究中心）；TCS Research（塔塔咨询服务公司研究部）
Table-top Rearrangement and Planning is a challenging problem that relies heavily on an excellent perception stack. The perception stack involves observing and registering the 3D scene on the table, detecting what objects are on the table, and how to manipulate them. Consequently, it greatly influences the system's task-planning and motion-planning stacks that follow. We present a comprehensive overview and discuss the different challenges associated with the perception module. This work is a result of our extensive involvement in the ICRA 2022 Open Cloud Robot Table Organization Challenge, in…
Robot manipulation in cluttered environments often requires complex and sequential rearrangement of multiple objects in order to achieve the desired reconfiguration of the target objects. Due to the sophisticated physical interactions involved in such scenarios, rearrangement-based manipulation is still limited to a small range of tasks and is especially vulnerable to physical uncertainties and perception noise. This paper presents a planning framework that leverages the efficiency of sampling-based planning approaches, and closes the manipulation loop by dynamically controlling the planning h…
We present a method to apply heuristic search algorithms to solve rearrangement planning by pushing problems. In these problems, a robot must push an object through clutter to achieve a goal. To do this, we exploit the fact that contact with objects in the environment is critical to goal achievement. We dynamically generate goal-directed primitives that create and maintain contact between robot and object at each state expansion during the search. These primitives focus exploration toward critical areas of state-space, providing tractability to the high-dimensional planning problem. We demonst…
KTH Royal Institute of Technology（瑞典皇家理工学院）；Yale University（耶鲁大学）
This paper addresses non-prehensile rearrangement planning problems where a robot is tasked to rearrange objects among obstacles on a planar surface. We present an efficient planning algorithm that is designed to impose few assumptions on the robot's non-prehensile manipulation abilities and is simple to adapt to different robot embodiments. For this, we combine sampling-based motion planning with reinforcement learning and generative modeling. Our algorithm explores the composite configuration space of objects and robot as a search over robot actions, forward simulated in a physics model. Thi…
Rearrangement-based nonprehensile manipulation still remains as a challenging problem due to the high-dimensional problem space and the complex physical uncertainties it entails. We formulate this class of problems as a coupled problem of local rearrangement and global action optimization by incorporating free-space transit motions between constrained rearranging actions. We propose a forest-based kinodynamic planning framework to concurrently search in multiple problem regions, so as to enable global exploration of the most task-relevant subspaces, while facilitating effective switches betwee…
Synchronized Dual-arm Rearrangement via Cooperative mTSP
National University of Defense Technology（国防科技大学）；Shenzhen University（深圳大学）；Xiangjiang Laboratory（湘江实验室）；Hunan University of Technology and Business（湖南工商大学）
Rutgers, the State University of New Jersey（罗格斯大学（新泽西州立大学））
We investigate the problem of coordinating two robot arms to solve non-monotone tabletop multi-object rearrangement tasks. In a non-monotone rearrangement task, complex object-object dependencies exist that require moving some objects multiple times to solve an instance. In working with two arms in a large workspace, some objects must be handed off between the robots, which further complicates the planning process. For the challenging dual-arm tabletop rearrangement problem, we develop effective task planning algorithms for scheduling the pick-n-place sequence that can be properly distributed …
Many real-world tasks, from house-cleaning to cooking, can be formulated as multi-object rearrangement problems -- where an agent needs to get specific objects into appropriate goal states. For such problems, we focus on the setting that assumes a pre-specified goal state, availability of perfect manipulation and object recognition capabilities, and a static map of the environment but unknown initial location of objects to be rearranged. Our goal is to enable home-assistive intelligent agents to efficiently plan for rearrangement under such partial observability. This requires efficient trade-…
National Institute of Advanced Industrial Science and Technology (AIST)（日本国立产业技术综合研究所（AIST））；Osaka University（大阪大学）
This paper develops a planner to find an optimal assembly sequence to assemble several objects. The input to the planner is the mesh models of the objects, the relative poses between the objects in the assembly, and the final pose of the assembly. The output is an optimal assembly sequence, namely (1) in which order should one assemble the objects, (2) from which directions should the objects be dropped, and (3) candidate grasps of each object. The proposed planner finds the optimal solution by automatically permuting, evaluating, and searching the possible assembly sequences considering stabi…
Osaka University（大阪大学）；National Inst. of AIST（国立先进工业科学技术研究所）
Planning dual-arm assembly of more than three objects is a challenging Task and Motion Planning (TAMP) problem. The assembly planner shall consider not only the pose constraints of objects and robots, but also the gravitational constraints that may break the finished part. This paper proposes a planner to plan the dual-arm assembly of more than three objects. It automatically generates the grasp configurations and assembly poses, and simultaneously searches and backtracks the grasp space and assembly space to accelerate the motion planning of robot arms. Meanwhile, the proposed method consider…
Solving task planning problems involving multiple objects and multiple robotic arms poses scalability challenges. Such problems involve not only coordinating multiple high-DoF arms, but also searching through possible sequences of actions including object placements, and handoffs. The current work identifies a useful connection between multi-arm rearrangement and recent results in multi-body path planning on graphs with vertex capacity constraints. Solving a synchronized multi-arm rearrangement at a high-level involves reasoning over a modal graph, where nodes correspond to stable object place…
NVIDIA（英伟达）；University of California San Diego（加州大学圣地亚哥分校）；University of Washington（华盛顿大学）
Robots will be expected to manipulate a wide variety of objects in complex and arbitrary ways as they become more widely used in human environments. As such, the rearrangement of objects has been noted to be an important benchmark for AI capabilities in recent years. We propose NeRP (Neural Rearrangement Planning), a deep learning based approach for multi-step neural object rearrangement planning which works with never-before-seen objects, that is trained on simulation data, and generalizes to the real world. We compare NeRP to several naive and model-based baselines, demonstrating that our ap…
This paper considers the problem of rearrangement planning, i.e finding a sequence of manipulation actions that displace multiple objects from an initial configuration to a given goal configuration. Rearrangement is a critical skill for robots so that they can effectively operate in confined spaces that contain clutter. Examples of tasks that require rearrangement include packing objects inside a bin, wherein objects need to lay according to a predefined pattern. In tight bins, collision-free grasps are often unavailable. Nonprehensile actions, such as pushing and sliding, are preferred becaus…
We address the problem of visually guided rearrangement planning with many movable objects, i.e., finding a sequence of actions to move a set of objects from an initial arrangement to a desired one, while relying on visual inputs coming from an RGB camera. To do so, we introduce a complete pipeline relying on two key contributions. First, we introduce an efficient and scalable rearrangement planning method, based on a Monte-Carlo Tree Search exploration strategy. We demonstrate that because of its good trade-off between exploration and exploitation our method (i) scales well with the number of…
Robotic Test Tube Rearrangement Using Combined Reinforcement Learning and Motion Planning
Graduate School of Engineering Science, Osaka University（大阪大学工学研究科）；H.U. Group Research Institute G.K.（H.U.集团综合研究所）；National Inst. of AIST（日本产业技术综合研究所）
Tel-Aviv University（特拉维夫大学）；Technion（以色列理工学院）；University of California, Berkeley（加州大学伯克利分校）
To compute robust 2D assembly plans, we present an approach that combines geometric planning with a deep neural network. We train the network using the Box2D physics simulator with added stochastic noise to yield robustness scores--the success probabilities of planned assembly motions. As running a simulation for every assembly motion is impractical, we train a convolutional neural network to map assembly operations, given as an image pair of the subassemblies before and after they are mated, to a robustness score. The neural network prediction is used within a planner to quickly prune out mot…
University of Colorado Boulder（科罗拉多大学博尔德分校）；University of Utah（犹他大学）
The sequence in which a complex product is assembled directly impacts the ease and efficiency of the assembly process, whether executed by a human or a robot. A sequence that gives the assembler the greatest freedom of movement is therefore desirable. Our main contribution is an expression of obstruction relationships between parts as a disassembly interference graph (DIG). We validate this heuristic by developing a disassembly sequence planner that partitions assemblies in a way that prioritizes access to parts, resulting in plans that are comparable in efficiency to two state-of-the-art asse…
Osaka University（大阪大学）；University of California, Santa Cruz（加州大学圣克鲁兹分校）
This paper develops a robotic manipulation planner for human-robot collaborative assembly. Unlike previous methods which study an independent and fully AI-equipped autonomous system, this paper explores the subtask distribution between a robot and a human and studies a human-in-the-loop robotic system for collaborative assembly. The system distributes the subtasks of an assembly to robots and humans by exploiting their advantages and avoiding their disadvantages. The robot in the system will work on pick-and-place tasks and provide workpieces to humans. The human collaborator will work on fine…
The authors propose the implementation of hybrid Fuzzy Logic-Genetic Algorithm (FL-GA) methodology to plan the automatic assembly and disassembly sequence of products. The GA-Fuzzy Logic approach is implemented onto two levels. The first level of hybridization consists of the development of a Fuzzy controller for the parameters of an assembly or disassembly planner based on GAs. This controller acts on mutation probability and crossover rate in order to adapt their values dynamically while the algorithm runs. The second level consists of the identification of theoptimal assembly or disassembly…
Siemens Corporate Technology（西门子中央研究院）；University of Freiburg（弗莱堡大学）
In automated manufacturing, robots must reliably assemble parts of various geometries and low tolerances. Ideally, they plan the required motions autonomously. This poses a substantial challenge due to high-dimensional state spaces and non-linear contact-dynamics. Furthermore, object poses and model parameters, such as friction, are not exactly known and a source of uncertainty. The method proposed in this paper models the task of parts assembly as a belief space planning problem over an underlying impedance-controlled, compliant system. To solve this planning problem we introduce an asymptoti…
Osaka University（大阪大学）；National Inst. of AIST（国立先进工业科学技术研究所）
This paper develops model-based grasp planning algorithms for assembly tasks. It focuses on industrial end-effectors like grippers and suction cups, and plans grasp configurations considering CAD models of target objects. The developed algorithms are able to stably plan a large number of high-quality grasps, with high precision and little dependency on the quality of CAD models. The undergoing core technique is superimposed segmentation, which pre-processes a mesh model by peeling it into facets. The algorithms use superimposed segments to locate contact points and parallel facets, and synthes…
