To enable a mobile manipulator to perform human tasks from a single teaching demonstration is vital to flexible manufacturing. We call our proposed method MMPA (Mobile Manipulator Process Automation with One-shot Teaching). Currently, there is no effective and robust MMPA framework which is not influenced by harsh industrial environments and the mobile base's parking precision. The proposed MMPA framework consists of two stages: collecting data (mobile base's location, environment information, end-effector's path) in the teaching stage for robot learning; letting the end-effector repeat the nearly same path as the reference path in the world frame to reproduce the work in the automation stage. More specifically, in the automation stage, the robot navigates to the specified location without the need of a precise parking. Then, based on colored point cloud registration, the proposed IPE (Iterative Pose Estimation by Eye & Hand) algorithm could estimate the accurate 6D relative parking pose of the robot arm base without the need of any marker. Finally, the robot could learn the error compensation from the parking pose's bias to modify the end-effector's path to make it repeat a nearly same path in the world coordinate system as recorded in the teaching stage. Hundreds of trials have been conducted with a real mobile manipulator to show the superior robustness of the system and the accuracy of the process automation regardless of the harsh industrial conditions and parking precision. For the released code, please contact marketing@amigaga.com
A Practical Roadmap to Learning from Demonstration for Robotic Manipulators in Manufacturing
University of Luxembourg（卢森堡大学）；Interdisciplinary Centre for Security, Reliability, and Trust (SnT)（安全、可靠性与跨学科信任中心（SnT））；Faculty of Science, Technology, and Medicine（科学、技术与医学学院）
Revolutionizing Battery Disassembly: The Design and Implementation of a Battery Disassembly Autonomous Mobile Manipulator Robot(BEAM-1)
School of Mechanical Engineering, Shanghai Jiao Tong University（上海交通大学机械工程学院）；Intel Labs China（英特尔中国实验室）；Faculty of Mechanical and Electrical Engineering, Kunming University of Science and Technology（昆明理工大学机电工程学院）；Beijing-Dublin International College, Beijing University of Technology（北京工业大学北京-都柏林国际学院）
Chongqing University of Posts and Telecommunications（重庆邮电大学）；Swansea University（斯旺西大学）
Over the past decades, industrial manipulators play a vital role in in various fields, like aircraft manufacturing and automobile manufacturing. However, an industrial manipulator without calibration suffers from its low absolute positioning accuracy, which extensively restricts its application in high-precision intelligent manufacture. Recent manipulator calibration methods are developed to address this issue, while they frequently encounter long-tail convergence and low calibration accuracy. To address this thorny issue, this work proposes a novel manipulator calibration method incorporating…
University of Southern California（南加州大学）；National Institute of Standards and Technology（美国国家标准与技术研究院）
Mobile manipulators can be used for machine tending and material handling tasks in small volume manufacturing applications. These applications usually have semi-structured work environment. The use of a fully autonomous mobile manipulator for such applications can be risky, as an inaccurate model of the workspace may result in damage to expensive equipment. On the other hand, the use of a fully teleoperated mobile manipulator may require a significant amount of operator time. In this paper, a semi-autonomous mobile manipulator is developed for safely and efficiently carrying out machine tendin…
Massachusetts Institute of Technology（麻省理工学院）；Texas A&M University（德克萨斯A&M大学）
The current dominant paradigm for robotic manipulation involves two separate stages: manipulator design and control. Because the robot's morphology and how it can be controlled are intimately linked, joint optimization of design and control can significantly improve performance. Existing methods for co-optimization are limited and fail to explore a rich space of designs. The primary reason is the trade-off between the complexity of designs that is necessary for contact-rich tasks against the practical constraints of manufacturing, optimization, contact handling, etc. We overcome several of the…
University of California, Berkeley（加州大学伯克利分校）；Honda Research Institute, USA（本田美国研究院）
A fundamental challenge in manipulating fabric for clothes folding and textiles manufacturing is computing "pick points" to effectively modify the state of an uncertain manifold. We present a supervised deep transfer learning approach to locate pick points using depth images for invariance to color and texture. We consider the task of bed-making, where a robot sequentially grasps and pulls at pick points to increase blanket coverage. We perform physical experiments with two mobile manipulator robots, the Toyota HSR and the Fetch, and three blankets of different colors and textures. We compare …
Czech Technical University in Prague（布拉格捷克理工大学）；New York University（纽约大学）；University of Pennsylvania（宾夕法尼亚大学）
Mobile manipulators have the potential to revolutionize modern agriculture, logistics and manufacturing. In this work, we present the design of a ground-based mobile manipulator for automated structure assembly. The proposed system is capable of autonomous localization, grasping, transportation and deployment of construction material in a semi-structured environment. Special effort was put into making the system invariant to lighting changes, and not reliant on external positioning systems. Therefore, the presented system is self-contained and capable of operating in outdoor and indoor conditi…
Fiber Bragg Grating (FBG) has shown great potential in shape and force sensing of continuum manipulators (CM) and biopsy needles. In the recent years, many researchers have studied different manufacturing and modeling techniques of FBG-based force and shape sensors for medical applications. These studies mainly focus on obtaining shape and force information in a static (or quasi-static) environment. In this paper, however, we study and evaluate dynamic environments where the FBG data is affected by vibration caused by a harmonic force e.g. a rotational debriding tool harmonically exciting the …
College of Engineering, University of Massachusetts Amherst（工程学院，马萨诸塞大学阿姆赫斯特分校）；College of Information and Computer Sciences, University of Massachusetts Amherst（信息与计算机科学学院，马萨诸塞大学阿姆赫斯特分校）
RobotDiffuse: Diffusion-Based Motion Planning for Redundant Manipulators with the ROP Obstacle Avoidance Dataset
School of Computer Science and Engineering, Beihang University, Beijing, China（北京航空航天大学计算机科学与工程学院）；School of Software, Beihang University, Beijing, China（北京航空航天大学软件学院）；Hangzhou Innovation Institute, Beihang University, Hangzhou, China（北京航空航天大学杭州创新研究院）
Fraunhofer IOSB（弗劳恩霍夫IOSB研究所）；Fraunhofer Institute of Optronics, System Technologies and Image Exploitation（弗劳恩霍夫光电、系统技术与图像开发研究所）
In order to enable physical human-robot interaction where humans and (mobile) manipulators share their workspace and work together, robots have to be equipped with important capabilities to guarantee human safety. The robots have to recognize possible collisions with the human co-worker and react anticipatorily by adapting their motion to avert dangerous situations while they are executing their task. Therefore, methods have been developed that allow to monitor the workspace of mobile manipulators using multiple depth sensors to gather information about the robot environment. This encompasses …
Fast and precise motion control is important for industrial robots in manufacturing applications. However, some collaborative robots sacrifice precision for safety, particular for high motion speed. The performance degradation is caused by the inability of the joint servo controller to address the uncertain nonlinear dynamics of the robot arm, e.g., due to joint flexibility. We consider two approaches to improve the trajectory tracking performance through feedforward compensation. The first approach uses iterative learning control, with the gradient-based iterative update generated from the ro…
This paper presents a physical interface for collaborative mobile manipulators in industrial manufacturing and logistics applications. The proposed work builds on our earlier MOCA-MAN interface, through which an operator could be physically coupled to a mobile manipulator to be assisted in performing daily activities. The previous interface was based on a magnetic clamp attached to one arm of the user for the coupling stage, and a bracelet based on EMG sensors on the other arm for human-robot communication via gestures. The new interface instead presents the following additions: i) An industri…
Ecole des Mines de Nantes（南特矿业学院）；Institut de Recherches en Communications et en Cybernétique de Nantes（南特通信与控制论研究所）；University of Agder（阿格德大学）
The paper presents the compliance errors compensation technique for over-constrained parallel manipulators under external and internal loadings. This technique is based on the non-linear stiffness modeling which is able to take into account the influence of non-perfect geometry of serial chains caused by manufacturing errors. Within the developed technique, the deviation compensation reduces to an adjustment of a target trajectory that is modified in the off-line mode. The advantages and practical significance of the proposed technique are illustrated by an example that deals with groove milli…
Robots are widely used in industry due to their efficiency and high accuracy in performance. One of the most intriguing issues in manufacturing stage of production line is to minimize significantly high percentage of energy consumed by these robot manipulators. The energy optimal control of robotic manipulators is a complex problem, as it requires a deep understanding of the robot's kinematics and dynamic behaviors. This paper propose a novel method of energy efficient path planning of an industrial robot arm in a workspace with multiple obstacles using differential evolution (DE) algorithm. T…
Osaka University（大阪大学）；National Institute of Advanced Industrial Science and Technology (AIST)（日本产业技术综合研究所）
Industrial robot manipulators are playing a more significant role in modern manufacturing industries. Though peg-in-hole assembly is a common industrial task which has been extensively researched, safely solving complex high precision assembly in an unstructured environment remains an open problem. Reinforcement Learning (RL) methods have been proven successful in solving manipulation tasks autonomously. However, RL is still not widely adopted on real robotic systems because working with real hardware entails additional challenges, especially when using position-controlled manipulators. The ma…
NVIDIA（英伟达）；Carnegie Mellon University（卡内基梅隆大学）；University of Washington（华盛顿大学）
In many applications, including logistics and manufacturing, robot manipulators operate in semi-structured environments alongside humans or other robots. These environments are largely static, but they may contain some movable obstacles that the robot must avoid. Manipulation tasks in these applications are often highly repetitive, but require fast and reliable motion planning capabilities, often under strict time constraints. Existing preprocessing-based approaches are beneficial when the environments are highly-structured, but their performance degrades in the presence of movable obstacles, …
Ecole des Mines de Nantes（南特矿业学院）；Institut de Recherche en Communications et Cybernétique de Nantes（南特通信与控制论研究所）
The paper is devoted to the geometrical calibration of industrial robots employed in precise manufacturing. To identify geometric parameters, an advanced calibration technique is proposed that is based on the non-linear experiment design theory, which is adopted for this particular application. In contrast to previous works, the calibration experiment quality is evaluated using a concept of the user-defined test-pose. In the frame of this concept, the related optimization problem is formulated and numerical routines are developed, which allow user to generate optimal set of manipulator configu…
SOFTMAP: Sim2Real Soft Robot Forward Modeling via Topological Mesh Alignment and Physics Prior
Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）；Bosch Center for Artificial Intelligence（博世人工智能中心）；School of Computer Science, The University of Sydney（悉尼大学计算机科学学院）；Australian Centre for Robotics, The University of Sydney（悉尼大学机器人研究中心）；College of Connected Computing, Vanderbilt University（范德比尔特大学连接计算学院）
本文提出SOFTMAP框架，结合拓扑对齐、轻量前向模型、残差修正网络和线性校准层，实现软手指 manipulator 的高精度 3D 前向建模与控制，取得毫米级轨迹跟踪和任务成功率提升。
Individualized manufacturing of cars requires kitting: the collection of individual sets of part variants for each car. This challenging logistic task is frequently performed manually by warehouseman. We propose a mobile manipulation robotic system for autonomous kitting, building on the Kuka Miiwa platform which consists of an omnidirectional base, a 7 DoF collaborative iiwa manipulator, cameras, and distance sensors. Software modules for detection and pose estimation of transport boxes, part segmentation in these containers, recognition of part variants, grasp generation, and arm trajectory …
Industrial robots are widely used in various manufacturing environments due to their efficiency in doing repetitive tasks such as assembly or welding. A common problem for these applications is to reach a destination without colliding with obstacles or other robot arms. Commonly used sampling-based path planning approaches such as RRT require long computation times, especially in complex environments. Furthermore, the environment in which they are employed needs to be known beforehand. When utilizing the approaches in new environments, a tedious engineering effort in setting hyperparameters ne…
Fused deposition modeling (FDM) using mobile robots instead of the gantry-based 3D printer enables additive manufacturing at a larger scale with higher speed. This introduces challenges including accurate localization, control of the printhead, and design of a stable mobile manipulator with low vibrations and proper degrees of freedom. We proposed and developed a low-cost non-holonomic mobile 3D printing system guided by a projector via learning-based visual servo-ing. It requires almost no manual calibration of the system parameters. Using a regular top-down projector without any expensive ex…
While rigid robots are extensively used in various applications, they are limited in the tasks they can perform and can be unsafe in close human-robot interactions. Soft robots on the other hand surpass the capabilities of rigid robots in several ways, such as compatibility with the work environments, degrees of freedom, manufacturing costs, and safe interactions with the environment. This thesis studies the behavior of Fiber Reinforced Elastomeric Enclosures (FREEs) as a particular type of soft pneumatic actuator that can be used in soft manipulators. A dynamic lumped-parameter model is creat…
New York University（纽约大学）；fortiss - Research Institute of the Free State of Bavaria（巴伐利亚自由州fortiss研究所）；New York University Abu Dhabi（纽约大学阿布扎比校区）；Technical University of Munich（慕尼黑工业大学）
The system design and algorithm development of mobile 3D printing robots need a realistic simulation. They require a mobile robot simulation platform to interoperate with a physics-based material simulation for handling interactions between the time-variant deformable 3D printing materials and other simulated rigid bodies in the environment, which is not available for roboticists yet. To bridge this gap and enable the real-time simulation of mobile 3D printing processes, we develop a simulation framework that includes particle-based viscoelastic fluid simulation and particle-to-mesh conversion…
INESC Porto（INESC 波尔图研究所）；University of Porto（波尔图大学）；University of Coimbra（科英布拉大学）
In this paper, an adaptive and low-cost robotic coating platform for small production series is presented. This new platform presents a flexible architecture that enables fast/automatic system adaptive behaviour without human intervention. The concept is based on contactless technology, using artificial vision and laser scanning to identify and characterize different workpieces travelling on a conveyor. Using laser triangulation, the workpieces are virtually reconstructed through a simplified cloud of three-dimensional (3D) points. From those reconstructed models, several algorithms are implem…
SPI-BoTER: Error Compensation for Industrial Robots via Sparse Attention Masking and Hybrid Loss with Spatial-Physical Information
School of Software（软件学院）；Northwestern Polytechnical University（西北工业大学）；School of Computer Science and Technology（计算机科学与技术学院）；University of Science and Technology of China（中国科学技术大学）；Suzhou Key Lab of Aerospace Industry Software and Data Science（苏州航空航天软件与数据科学重点实验室）；Yangtze River Delta Research Institute of NPU（南京航空航天大学长江三角洲研究院）
