Multiple mobile manipulators show superiority in the tasks requiring mobility and dexterity compared with a single robot, especially when manipulating/transporting bulky objects. However, closed-chain of the system, redundancy of each mobile manipulator and obstacles in the environment bring challenges to the motion planning problem. In this paper, we propose a novel semi-coupled hierarchical framework (SCHF), which decomposes the problem into two semi-coupled sub-problems.To be specific, the centralized layer plans the object's motion first and then the decentralized layer independently explores the redundancy of each robot in real-time. A notable feature is that the lower bound of the redundancy constraint metric is ensured besides the closed-chain and obstacle-avoidance constraints in the centralized layer, which ensures the object's motion can be executed by each robot in the decentralized layer. Simulated results show that the success rate and time cost of SCHF outperforms the fully centralized planner and fully decoupled hierarchical planner significantly. In addition, cluttered real-world experiments also show the feasibility of the SCHF in the transportation tasks. A video clip in various scenarios can be found at https://youtu.be/Y8ZrnspIuBg.
University of Modena and Reggio Emilia（摩德纳-雷焦艾米利亚大学）
Human-robot collaborative tasks foresee interactions between humans and robots with various degrees of complexity. Specifically, for tasks which involve physical contact among the agents, challenges arise in the modelling and control of such interaction. In this paper we propose a control architecture capable of ensuring a flexible and robustly stable physical human-robot interaction, focusing on a collaborative transportation task. The architecture is deployed onto a mobile manipulator, modelled as a whole-body structure, which aids the operator during the transportation of an unwieldy load. …
This paper addresses the problem of cooperative object transportation for multiple Underwater Vehicle Manipulator Systems (UVMSs) in a constrained workspace involving static obstacles. We propose a Nonlinear Model Predictive Control (NMPC) approach for a team of UVMSs in order to transport an object while avoiding significant constraints and limitations such as: kinematic and representation singularities, obstacles within the workspace, joint limits and control input saturations. More precisely, by exploiting the coupled dynamics between the robots and the object, and using certain load sharin…
Keep it Upright: Model Predictive Control for Nonprehensile Object Transportation with Obstacle Avoidance on a Mobile Manipulator
Technical University of Munich（慕尼黑工业大学）；University of Toronto Institute for Aerospace Studies（多伦多大学航空航天研究所）；University of Toronto Robotics Institute（多伦多大学机器人研究所）；Munich Institute of Robotics and Machine Intelligence（慕尼黑机器人与机器智能研究所）；Vector Institute for Artificial Intelligence（矢量人工智能研究所）
Tsinghua University（清华大学）；Beijing Institute of Control Engineering（北京控制工程研究所）；City University of Hong Kong（香港城市大学）
Recent years have seen the emergence of non-cooperative objects in space, like failed satellites and space junk. These objects are usually operated or collected by free-float dual-arm space manipulators. Thanks to eliminating the difficulties of modeling and manual parameter-tuning, reinforcement learning (RL) methods have shown a more promising sign in the trajectory planning of space manipulators. Although previous studies demonstrate their effectiveness, they cannot be applied in tracking dynamic targets with unknown rotation (non-cooperative objects). In this paper, we proposed a learning …
ACCESS Linnaeus Center（ACCESS林奈中心）；School of Electrical Engineering, KTH Royal Institute of Technology（瑞典皇家理工学院电气工程学院）；KTH Center for Autonomous Systems（瑞典皇家理工学院自主系统中心）
This paper addresses the problem of cooperative transportation of an object rigidly grasped by N robotic agents. We propose a Nonlinear Model Predictive Control (NMPC) scheme that guarantees the navigation of the object to a desired pose in a bounded workspace with obstacles, while complying with certain input saturations of the agents. The control scheme is based on inter-agent communication and is decentralized in the sense that each agent calculates its own control signal. Moreover, the proposed methodology ensures that the agents do not collide with each other or with the workspace obstacl…
Motion Planning of Cooperative Nonholonomic Mobile Manipulators
Department of Mechanical Engineering, Indian Institute of Technology Bombay（印度理工学院班加罗尔机械工程系）；Center for Systems and Control, Indian Institute of Technology Bombay（印度理工学院班加罗尔系统控制中心）
KTH Royal Institute of Technology（KTH皇家理工学院）；School of Electrical Engineering and Computer Science（电气工程与计算机科学学院）
Control and planning of multi-agent systems is an active and increasingly studied topic of research, with many practical applications such as rescue missions, security, surveillance, and transportation. This thesis addresses the planning and control of multi-agent systems under temporal logic tasks. The considered systems concern complex, robotic, manipulator-endowed systems, which can coordinate in order to execute complicated tasks, including object manipulation/transportation. Motivated by real-life scenarios, we take into account high-order dynamics subject to model uncertainties and unkno…
IIIT Hyderabad（国际信息技术研究所海得拉巴分校）；University of Genoa（热那亚大学）；University of Tübingen（图宾根大学）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；IIT Jodhpur（焦特布尔印度理工学院）
In this paper, a kinematic motion planning algorithm for cooperative spatial payload manipulation is presented. A hierarchical approach is introduced to compute real-time collision-free motion plans for a formation of mobile manipulator robots. Initially, collision-free configurations of a deformable 2-D virtual bounding box are identified, over a planning horizon, to define a convex workspace for the entire system. Then, 3-D payload configurations whose projections lie within the defined convex workspace are computed. Finally, a convex decentralized model-predictive controller is formulated t…
The University of Sydney（悉尼大学）；Cohda Wireless（科达无线公司）
The paper addresses the vehicle-to-X (V2X) data fusion for cooperative or collective perception (CP). This emerging and promising intelligent transportation systems (ITS) technology has enormous potential for improving efficiency and safety of road transportation. Recent advances in V2X communication primarily address the definition of V2X messages and data dissemination amongst ITS stations (ITS-Ss) in a traffic environment. Yet, a largely unsolved problem is how a connected vehicle (CV) can efficiently and consistently fuse its local perception information with the data received from other I…
The University of Sydney（悉尼大学）；Cohda Wireless（科达无线）
Cooperative perception, or collective perception (CP) is an emerging and promising technology for intelligent transportation systems (ITS). It enables an ITS station (ITS-S) to share its local perception information with others by means of vehicle-to-X (V2X) communication, thereby achieving improved efficiency and safety in road transportation. In this paper, we present our recent progress on the development of a connected and automated vehicle (CAV) and intelligent roadside unit (IRSU). We present three different experiments to demonstrate the use of CP service within intelligent infrastructu…
Robust Nonprehensile Object Transportation with Uncertain Inertial Parameters
Technical University of Munich（慕尼黑工业大学）；University of Toronto（多伦多大学）；University of Toronto Institute for Aerospace Studies（多伦多大学航空航天研究所）；University of Toronto Robotics Institute（多伦多大学机器人研究所）；Munich Institute of Robotics and Machine Intelligence（慕尼黑机器人与机器智能研究所）；Vector Institute for Artificial Intelligence（向量人工智能研究所）
Towards the Automation in the Space Station: Feasibility Study and Ground Tests of a Multi-Limbed Intra-Vehicular Robot
Japan Aerospace Exploration Agency (JAXA)（日本宇宙航空研究开发机构）；Space Robotics Lab (SRL)（空间机器人实验室）；Department of Aerospace Engineering, Graduate School of Engineering, Tohoku University（工学研究生院航空工程系）；Hamano Products Co., Ltd（Hamano公司）
Physics Residual Dynamics and Reduced Order Whole-Body Planning for Obstacle Aware Human Robot Cloth CoTransportation
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)（穆罕默德·本·扎耶德人工智能大学）；Technical University of Munich (TUM)（慕尼黑工业大学）；Tampere University（坦佩雷大学）；Aalto University（阿尔托大学）
Socially-aware Object Transportation by a Mobile Manipulator in Static Planar Environments with Obstacles
Computer Vision and Robotics Laboratory (VeRLab), Department of Computer Science, Universidade Federal de Minas Gerais（计算机视觉与机器人实验室（VeRLab）、计算机科学系、巴西矿务大学）
This paper addresses the problem of cooperative object transportation for multiple Underwater Vehicle Manipulator Systems (UVMSs) in a constrained workspace with static obstacles, where the coordination relies solely on implicit communication arising from the physical interaction of the robots with the commonly grasped object. We propose a novel distributed leader-follower architecture, where the leading UVMS, which has knowledge of the object's desired trajectory, tries to achieve the desired tracking behavior via an impedance control law, navigating in this way, the overall formation towards…
Design, implementation, and machine learning issues associated with developing a control system for a serpentine robotic manipulator are explored. The controller developed provides autonomous control of the serpentine robotic manipulatorduring operation of the manipulator within an enclosed environment such as an underground storage tank. The controller algorithms make use of both low-level joint angle control employing force/position feedback constraints, and high-level coordinated control of end-effector positioning. This approach has resulted in both high-level full robotic control and low-…
School of Electrical & Computer Engineering, Univ. of Georgia（佐治亚大学电气与计算机工程学院）；Dept. of Electrical & Computer Engineering, Univ. of Central Florida（中佛罗里达大学电气与计算机工程系）
Cooperative driving relies on communication among vehicles to create situational awareness. One application of cooperative driving is Cooperative Adaptive Cruise Control (CACC) that aims at enhancing highway transportation safety and capacity. Model-based communication (MBC) is a new paradigm with a flexible content structure for broadcasting joint vehicle-driver predictive behavioral models. The vehicle's complex dynamics and diverse driving behaviors add complexity to the modeling process. Gaussian process (GP) is a fully data-driven and non-parametric Bayesian modeling approach which can be…
Cooperative Transportation using Multiple Single-Rotor Robots and Decentralized Control for Unknown Payloads
Cloud Informatics Research-Domain, Toyota Central R&D Labs., Inc.（丰田中央研究实验室云信息研究部门）；R-Frontier Division, Frontier Research Center. Toyota Motor Corporation（丰田电机公司前沿研究中心R前沿部门）
New York University（纽约大学）；New York University Abu Dhabi（纽约大学阿布扎比分校）
The control problem of a multi-copter swarm, mechanically coupled through a modular lattice structure of connecting rods, is considered in this article. The system's structural elasticity is considered in deriving the system's dynamics. The devised controller is robust against the induced flexibilities, while an inherent adaptation scheme allows for the control of asymmetrical configurations and the transportation of unknown payloads. Certain optimization metrics are introduced for solving the individual agent thrust allocation problem while achieving maximum system flight time, resulting in a…
The problem of self-tuning control of cooperative manipulators forming a closed kinematic chain in the presence of an inaccurate kinematics model is addressed using adaptive machine learning. The kinematic parameters pertaining to the relative position/orientation uncertainties of the interconnected manipulators are updated online by two cascaded estimators in order to tune a cooperative controller for achieving accurate motion tracking with minimum-norm actuation force. This technique permits accurate calibration of the relative kinematics of the involved manipulators without needing high pre…
This paper studies the time-optimal path tracking problem for a team of cooperating robotic manipulators carrying an object. Considering the problem for rigidly grasped objects, we show that it can be cast as a convex optimization problem and solved efficiently with a guarantee of optimality. When formulating the problem, we avoid using a particular wrench distribution and exploit the full actuation available to the system. Then, we consider the problem for grasps using frictional forces and show that this problem also, under a force-closure grasp assumption, can be formulated as a convex opti…
Research Institute for Trustworthy Autonomous Systems, Southern University of Science and Technology（南方科技大学可信自主系统研究院）；University of Macau（澳门大学）；Singapore University of Technology and Design（新加坡科技设计大学）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of New South Wales（新南威尔士大学）
This paper investigates a fully distributed cooperation scheme for networked mobile manipulators. To achieve cooperative task allocation in a distributed way, an adaptation-based estimation law is established for each robotic agent to estimate the desired local trajectory. In addition, wrench synthesis is analyzed in detail to lay a solid foundation for tight cooperation tasks. Together with the estimated task, a set of distributed adaptive controllers is proposed to achieve motion synchronization of the mobile manipulator ensemble over a directed graph with a spanning tree irrespective of the…
Universidade Federal de Minas Gerais（米纳斯吉拉斯联邦大学）；ESRI Inc.（环境系统研究所公司）
This paper presents a novel methodology that allows a swarm of robots to perform a cooperative transportation task. Our approach consists of modeling the swarm as a {\em Gibbs Random Field} (GRF), taking advantage of this framework's locality properties. By setting appropriate potential functions, robots can dynamically navigate, form groups, and perform cooperative transportation in a completely decentralized fashion. Moreover, these behaviors emerge from the local interactions without the need for explicit communication or coordination. To evaluate our methodology, we perform a series of sim…
University of Cincinnati（辛辛那提大学）；Harbin Institute of Technology（哈尔滨工业大学）；Tsinghua University（清华大学）
Object transportation could be a challenging problem for a single robot due to the oversize and/or overweight issues. A multi-robot system can take the advantage of increased driving power and more flexible configuration to solve such a problem. However, increased number of individuals also changed the dynamics of the system which makes control of a multi-robot system more complicated. Even worse, if the whole system is sitting on a centralized decision making unit, the data flow could be easily overloaded due to the upscaling of the system. In this research, we propose a decentralized control…
Dept. Aerospace and Mech. Engineering, University of Notre Dame（美国圣母大学航空航天与机械工程系）；Dept. of Mech. Engineering, Massachusetts Institute of Technology（麻省理工学院机械工程系）
This paper studies collaboration through the cloud in the context of cooperative adaptive control for robot manipulators. We first consider the case of multiple robots manipulating a common object through synchronous centralized update laws to identify unknown inertial parameters. Through this development, we introduce a notion of Collective Sufficient Richness, wherein parameter convergence can be enabled through teamwork in the group. The introduction of this property and the analysis of stable adaptive controllers that benefit from it constitute the main new contributions of this work. Buil…
Univ. of Georgia（佐治亚大学）；Univ. of Central Florida（中佛罗里达大学）
Cooperative driving, enabled by Vehicle-to-Everything (V2X) communication, is expected to significantly contribute to the transportation system's safety and efficiency. Cooperative Adaptive Cruise Control (CACC), a major cooperative driving application, has been the subject of many studies in recent years. The primary motivation behind using CACC is to reduce traffic congestion and improve traffic flow, traffic throughput, and highway capacity. Since the information flow between cooperative vehicles can significantly affect the dynamics of a platoon, the design and performance of control compo…
PerceptIn（感知创新公司）；South China University of Technology（华南理工大学）；Northwestern University（西北大学）
The infrastructure-vehicle cooperative autonomous driving approach depends on the cooperation between intelligent roads and intelligent vehicles. This approach is not only safer but also more economical compared to the traditional on-vehicle-only autonomous driving approach. In this paper, we introduce our real-world deployment experiences of cooperative autonomous driving, and delve into the details of new challenges and opportunities. Specifically, based on our progress towards commercial deployment, we follow a three-stage development roadmap of the cooperative autonomous driving approach:i…
This paper considers a pursuit-evasion scenario among three agents -- an evader, a pursuer, and a defender. We design cooperative guidance laws for the evader and the defender team to safeguard the evader from an attacking pursuer. Unlike differential games, optimal control formulations, and other heuristic methods, we propose a novel perspective on designing effective nonlinear feedback control laws for the evader-defender team using a time-constrained guidance approach. The evader lures the pursuer on the collision course by offering itself as bait. At the same time, the defender protects th…
Czech Technical University in Prague（布拉格捷克理工大学）；New York University（纽约大学）；University of Pennsylvania（宾夕法尼亚大学）
Mobile manipulators have the potential to revolutionize modern agriculture, logistics and manufacturing. In this work, we present the design of a ground-based mobile manipulator for automated structure assembly. The proposed system is capable of autonomous localization, grasping, transportation and deployment of construction material in a semi-structured environment. Special effort was put into making the system invariant to lighting changes, and not reliant on external positioning systems. Therefore, the presented system is self-contained and capable of operating in outdoor and indoor conditi…
Self-Organizing Aerial Swarm Robotics for Resilient Load Transportation : A Table-Mechanics-Inspired Approach
School of Automation Science and Electrical Engineering, Beihang University（北京航空航天大学自动化科学与电气工程学院）；Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong（香港中文大学机械与自动化工程系）
University of Alicante（阿利坎特大学）；Northeastern University（东北大学）
Tasks in outdoor open world environments are now ripe for automation with mobile manipulators. The dynamic, unstructured and unknown environments associated with such tasks -- a prime example would be collecting roadside trash -- makes them particularly challenging. In this paper we present an approach to solving the problem of picking up, transporting, and dropping off novel objects outdoors. Our solution integrates a navigation system, a grasp detection and planning system, and a custom task planner. We perform experiments that demonstrate that the system can be used to transport a wide clas…
