This paper presents a planner that can automatically find an optimal assembly sequence for a dual-arm robot to assemble the soma blocks. The planner uses the mesh model of objects and the final state of the assembly to generate all possible assembly sequence and evaluate the optimal assembly sequence by considering the stability, graspability, assemblability, as well as the need for a second arm. Especially, the need for a second arm is considered when supports from worktables and other workpieces are not enough to produce a stable assembly. The planner will refer to an assisting grasp to additionally hold and support the unstable components so that the robot can further assemble new workpieces and finally reach a stable state. The output of the planner is the optimal assembly orders, candidate grasps, assembly directions, and the assisting grasps if any. The output of the planner can be used to guide a dual-arm robot to perform the assembly task. The planner is verified in both simulations and real-world executions.
Dual-arm Motion Generation for Repositioning Care based on Deep Predictive Learning with Somatosensory Attention Mechanism
Future Robotics Organization, Waseda University（早稻田大学未来机器人组织）；Microsoft Research Asia（微软亚洲研究院）；Faculty of Science and Engineering, Waseda University（早稻田大学理工学院）
Osaka University（大阪大学）；National Inst. of AIST（国立先进工业科学技术研究所）
Planning dual-arm assembly of more than three objects is a challenging Task and Motion Planning (TAMP) problem. The assembly planner shall consider not only the pose constraints of objects and robots, but also the gravitational constraints that may break the finished part. This paper proposes a planner to plan the dual-arm assembly of more than three objects. It automatically generates the grasp configurations and assembly poses, and simultaneously searches and backtracks the grasp space and assembly space to accelerate the motion planning of robot arms. Meanwhile, the proposed method consider…
Osaka University（大阪大学）；National Inst. of AIST（国立产业技术综合研究所）
This paper presents a manipulation planning algorithm for robots to reorient objects. It automatically finds a sequence of robot motion that manipulates and prepares an object for specific tasks. Examples of the preparatory manipulation planning problems include reorienting an electric drill to cut holes, reorienting workpieces for assembly, and reorienting cargo for packing, etc. The proposed algorithm could plan single and dual arm manipulation sequences to solve the problems. The mechanism under the planner is a regrasp graph which encodes grasp configurations and object poses. The algorith…
Robots are widely used in industry due to their efficiency and high accuracy in performance. One of the most intriguing issues in manufacturing stage of production line is to minimize significantly high percentage of energy consumed by these robot manipulators. The energy optimal control of robotic manipulators is a complex problem, as it requires a deep understanding of the robot's kinematics and dynamic behaviors. This paper propose a novel method of energy efficient path planning of an industrial robot arm in a workspace with multiple obstacles using differential evolution (DE) algorithm. T…
Bosch Center for Artificial Intelligence（博世人工智能中心）；Technical University of Berlin（柏林工业大学）；RWTH Aachen University（亚琛工业大学）
Automated bin-picking is a prerequisite for fully automated manufacturing and warehouses. To successfully pick an item from an unstructured bin the robot needs to first detect possible grasps for the objects, decide on the object to remove and consequently plan and execute a feasible trajectory to retrieve the chosen object. Over the last years significant progress has been made towards solving these problems. However, when multiple robot arms are cooperating the decision and planning problems become exponentially harder. We propose an integrated multi-arm bin-picking pipeline (IMAPIP), and de…
CAMP: Cooperative Arm-Hand Motion Planning in Constrained Spaces
Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；Great Bay University（大湾区大学）；China Pharmaceutical University（中国药科大学）；KTH Royal Institute of Technology（瑞典皇家理工学院）；The Hong Kong Polytechnic University（香港理工大学）；The University of Hong Kong（香港大学）；University of Belgrade（贝尔格莱德大学）
Automation applications are pushing the deployment of many high DoF manipulators in warehouse and manufacturing environments. This has motivated many efforts on optimizing manipulation tasks involving a single arm. Coordinating multiple arms for manipulation, however, introduces additional computational challenges arising from the increased DoFs, as well as the combinatorial increase in the available operations that many manipulators can perform, including handoffs between arms. The focus here is on the case of pick-and-place tasks, which require a sequence of handoffs to be executed, so as to…
Reflective VLM Planning for Dual-Arm Desktop Cleaning: Bridging Open-Vocabulary Perception and Precise Manipulation
Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）；Department of Electrical and Computer Engineering, Carnegie Mellon University（卡内基梅隆大学电气与计算机工程系）；College of Aeronautics & Engineering, Kent State University（肯特州立大学航空与工程学院）
We study the path planning problem for continuum-arm robots, in which we are given a starting and an end point, and we need to compute a path for the tip of the continuum arm between the two points. We consider both cases where obstacles are present and where they are not. We demonstrate how to leverage the continuum arm features to introduce a new model that enables a path planning approach based on the configurations graph, for a continuum arm consisting of three sections, each consisting of three muscle actuators. The algorithm we apply to the configurations graph allows us to exploit paral…
Robotic manipulation of tethered tools is widely seen in robotic work cells. They may cause excess strain on the tool's cable or undesired entanglements with the robot's arms. This paper presents a manipulation planner with cable orientation constraints for tethered tools suspended by tool balancers. The planner uses orientation constraints to limit the bending of the balancer's cable while the robot manipulates a tool and places it in a desired pose. The constraints reduce entanglements and decrease the torque induced by the cable on the robot joints. Simulation and real-world experiments sho…
Generating natural and physically feasible motions for legged robots has been a challenging problem due to its complex dynamics. In this work, we introduce a novel learning-based framework of autoregressive motion planner (ARMP) for quadruped locomotion and navigation. Our method can generate motion plans with an arbitrary length in an autoregressive fashion, unlike most offline trajectory optimization algorithms for a fixed trajectory length. To this end, we first construct the motion library by solving a dense set of trajectory optimization problems for diverse scenarios and parameter settin…
Solving task planning problems involving multiple objects and multiple robotic arms poses scalability challenges. Such problems involve not only coordinating multiple high-DoF arms, but also searching through possible sequences of actions including object placements, and handoffs. The current work identifies a useful connection between multi-arm rearrangement and recent results in multi-body path planning on graphs with vertex capacity constraints. Solving a synchronized multi-arm rearrangement at a high-level involves reasoning over a modal graph, where nodes correspond to stable object place…
Synchronized Dual-arm Rearrangement via Cooperative mTSP
National University of Defense Technology（国防科技大学）；Shenzhen University（深圳大学）；Xiangjiang Laboratory（湘江实验室）；Hunan University of Technology and Business（湖南工商大学）
Rutgers, the State University of New Jersey（罗格斯大学（新泽西州立大学））
We investigate the problem of coordinating two robot arms to solve non-monotone tabletop multi-object rearrangement tasks. In a non-monotone rearrangement task, complex object-object dependencies exist that require moving some objects multiple times to solve an instance. In working with two arms in a large workspace, some objects must be handed off between the robots, which further complicates the planning process. For the challenging dual-arm tabletop rearrangement problem, we develop effective task planning algorithms for scheduling the pick-n-place sequence that can be properly distributed …
Towards Proprioception-Aware Embodied Planning for Dual-Arm Humanoid Robots
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；AgiBot；School of Computer Science, Peking University（北京大学计算机学院）
The Hong Kong Polytechnic University（香港理工大学）；University of York（约克大学）；Shanghai Jiatong University（上海交通大学）
Most object manipulation strategies for robots are based on the assumption that the object is rigid (i.e., with fixed geometry) and the goal's details have been fully specified (e.g., the exact target pose). However, there are many tasks that involve spatial relations in human environments where these conditions may be hard to satisfy, e.g., bending and placing a cable inside an unknown container. To develop advanced robotic manipulation capabilities in unstructured environments that avoid these assumptions, we propose a novel long-horizon framework that exploits contrastive planning in findin…
Osaka University（大阪大学）；National Inst. of AIST（国立先进工业科学技术研究所）
This paper proposes a combined task and motion planner for a dual-arm robot to use a suction cup tool. The planner consists of three sub-planners -- A suction pose sub-planner and two regrasp and motion sub-planners. The suction pose sub-planner finds all the available poses for a suction cup tool to suck on the object, using the models of the tool and the object. The regrasp and motion sub-planner builds the regrasp graph that represents all possible grasp sequences to reorient and move the suction cup tool from an initial pose to a goal pose. Two regrasp graphs are used to plan for a single …
Dual-Arm Hierarchical Planning for Laboratory Automation: Vibratory Sieve Shaker Operations
College of Intelligence Science and Technology, and the National Key Laboratory of Equipment State Sensing and Smart Support, National University of Defense Technology（智能科学与技术学院，装备状态感知与智能支持国家重点实验室，国防科技大学）；College of Aerospace Science and Engineering, National University of Defense Technology（航空航天科学与工程学院，国防科技大学）
RoboPARA: Dual-Arm Robot Planning with Parallel Allocation and Recomposition Across Tasks
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, School of Artificial Intelligence, Beihang University（北京未来区块链与隐私计算先进创新中心，人工智能学院，北航）；Beijing Innovation Center of Humanoid Robotics（北京人形机器人创新中心）；Center for Applied Statistics, School of Statistics, Renmin University of China（应用统计中心，统计学院，中国人民大学）
University of Tsukuba（筑波大学）；RIKEN Brain Science Institute（日本理化学研究所脑科学研究所）；The University of Tokyo（东京大学）
We report on an extended robot control application of a contact-less and airborne ultrasonic tactile display (AUTD) stimulus-based brain-computer interface (BCI) paradigm, which received last year The Annual BCI Research Award 2014. In the award winning human communication augmentation paradigm the six palm positions are used to evoke somatosensory brain responses, in order to define a novel contactless tactile BCI. An example application of a small robot management is also presented in which the users control a small robot online.
Tsinghua University（清华大学）；University of California, Berkeley（加州大学伯克利分校）
Manipulating deformable linear objects (DLOs) to achieve desired shapes in constrained environments with obstacles is a meaningful but challenging task. Global planning is necessary for such a highly-constrained task; however, accurate models of DLOs required by planners are difficult to obtain owing to their deformable nature, and the inevitable modeling errors significantly affect the planning results, probably resulting in task failure if the robot simply executes the planned path in an open-loop manner. In this paper, we propose a coarse-to-fine framework to combine global planning and loc…
Multi-Objective Trajectory Planning for a Robotic Arm in Curtain Wall Installation
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Shenzhen University of Advanced Technology and University of Chinese Academy of Sciences（深圳先进技术大学和中国科学院大学）
Mechanical Engineering, University of Michigan（密歇根大学机械工程学院）；Robotics Institute, University of Michigan（密歇根大学机器人研究所）；Mechanical and Industrial Engineering, Louisiana State University（路易斯安那州立大学机械与工业工程学院）；Naval Architecture and Marine Engineering, University of Michigan（密歇根大学船舶与海洋工程学院）；Ford Motor Company（福特汽车公司）；Office of Naval Research（美国海军研究办公室）
For robotic arms to operate in arbitrary environments, especially near people, it is critical to certify the safety of their motion planning algorithms. However, there is often a trade-off between safety and real-time performance; one can either carefully design safe plans, or rapidly generate potentially-unsafe plans. This work presents a receding-horizon, real-time trajectory planner with safety guarantees, called ARMTD (Autonomous Reachability-based Manipulator Trajectory Design). The method first computes (offline) a reachable set of parameterized trajectories for each joint of an arm. Eac…
A Versatile Neural Network Configuration Space Planning and Control Strategy for Modular Soft Robot Arms
Biorobotics Institute and the Department of Excellence in Robotics and AI（生物机器人研究所和机器人与人工智能卓越部门）；Scuola Superiore Sant’Anna（圣安娜高等学院）；CREATE Lab（CREATE实验室）；EPFL（瑞士联邦理工学院）
Imperial College London（伦敦帝国理工学院）；University of Pennsylvania（宾夕法尼亚大学）
Robotic dual-arm twisting is a common but very challenging task in both industrial production and daily services, as it often requires dexterous collaboration, a large scale of end-effector rotating, and good adaptivity for object manipulation. Meanwhile, safety and efficiency are preliminary concerns for robotic dual-arm coordinated manipulation. Thus, the normally adopted fully automated task execution approaches based on environmental perception and motion planning techniques are still inadequate and problematic for the arduous twisting tasks. To this end, this paper presents a novel strate…
QuadHand: A Compact Quadrotor Aerial Manipulator with MRC-SDF-Based Whole-Body Motion Planning
Nanyang Technological University（南洋理工大学）；NTU–VinUni Joint Research Laboratory for Embodied AI and Robotics（南洋理工大学-维纳大学具身智能与机器人联合研究实验室）；VinUniversity（维纳大学）
Beijing Institute of Control Engineering（北京控制工程研究所）；City University of Hong Kong（香港城市大学）；Qian Xuesen Laboratory of Space Technology（钱学森空间技术实验室）；Tsinghua University（清华大学）
Reinforcement learning methods as a promising technique have achieved superior results in the motion planning of free-floating space robots. However, due to the increase in planning dimension and the intensification of system dynamics coupling, the motion planning of dual-arm free-floating space robots remains an open challenge. In particular, the current study cannot handle the task of capturing a non-cooperative object due to the lack of the pose constraint of the end-effectors. To address the problem, we propose a novel algorithm, EfficientLPT, to facilitate RL-based methods to improve plan…
Graduate School of Engineering Science, Osaka University（大阪大学工学研究科）；School of Automation, Guangdong University of Technology（广东工业大学自动化学院）
Planning a motion for inserting pegs remains an open problem. The difficulty lies in both the inevitable errors in the grasps of a robotic hand and absolute precision problems in robot joint motors. This paper proposes an integral method to solve the problem. The method uses combined task and motion planning to plan the grasps and motion for a dual-arm robot to pick up the objects and move them to assembly poses. Then, it controls the dual-arm robot using a compliant strategy (a combination of linear search, spiral search, and impedance control) to finish up the insertion. The method is implem…
Beijing Academy of Agriculture and Forestry Sciences（北京市农林科学院）；Jiangsu University（江苏大学）
The emergence of harvesting robotics offers a promising solution to the issue of limited agricultural labor resources and the increasing demand for fruits. Despite notable advancements in the field of harvesting robotics, the utilization of such technology in orchards is still limited. The key challenge is to improve operational efficiency. Taking into account inner-arm conflicts, couplings of DoFs, and dynamic tasks, we propose a task planning strategy for a harvesting robot with four arms in this paper. The proposed method employs a Markov game framework to formulate the four-arm robotic har…
Described here is a simple, reliable, and quite general method for rapid computation of robot arm inverse kinematic solutions and motion path plans in the presence of complex obstructions. The method derived from the MSC (map-seeking circuit) algorithm, optimized to exploit the characteristics of practical arm configurations. The representation naturally incorporates both arm and obstacle geometries. The consequent performance on modern hardware is suitable for applications requiring real-time response, including smooth continuous avoidance of dynamic obstacles which impinge on the planned pat…
This paper presents a combined task and motion planner for a robot arm to carry out 3D metal wire curving tasks by collaborating with a bending machine. We assume a collaborative robot that is safe to work in a human environment but has a weak payload to bend objects with large stiffness, and developed a combined planner for the robot to use a bending machine. Our method converts a 3D curve to a bending set and generates the feasible bending sequence, machine usage, robotic grasp poses, and pick-and-place arm motion considering the combined task and motion level constraints. Compared with prev…
Autonomous dual-arm manipulation is an essential skill to deploy robots in unstructured scenarios. However, this is a challenging undertaking, particularly in terms of perception and planning. Unstructured scenarios are full of objects with different shapes and appearances that have to be grasped in a very specific manner so they can be functionally used. In this paper we present an integrated approach to perform dual-arm pick tasks autonomously. Our method consists of semantic segmentation, object pose estimation, deformable model registration, grasp planning and arm trajectory optimization. …
Robotics Institute Carnegie Mellon University（卡内基梅隆大学机器人研究所）；Siemens CT（西门子中央研究院）
Applying intelligent robot arms in dynamic uncertain environments (i.e., flexible production lines) remains challenging, which requires efficient algorithms for real time trajectory generation. The motion planning problem for robot trajectory generation is highly nonlinear and nonconvex, which usually comes with collision avoidance constraints, robot kinematics and dynamics constraints, and task constraints (e.g., following a Cartesian trajectory defined on a surface and maintain the contact). The nonlinear and nonconvex planning problem is computationally expensive to solve, which limits the …
