We introduce a new approach to tackle the mobile manipulator task sequencing problem. We leverage computational geometry, graph theory and combinatorial optimization to yield a principled method to segment the task-space targets into clusters, analytically determine reachable base pose for each cluster, and find task sequences that minimize the number of base movements and robot execution time. By clustering targets first and by doing so from first principles, our solution is more general and computationally efficient when compared to existing methods.
Task-Space Clustering for Mobile Manipulator Task Sequencing
Singapore Centre for 3D Printing (SC3DP)（新加坡3D打印中心（SC3DP））；Nanyang Technological University (NTU)（南洋理工大学（NTU））；HP-NTU Digital Manufacturing Corporate Lab（惠普-南洋理工大学数字制造联合实验室）；Eureka Robotics（优傲机器人（Eureka Robotics））
University College London（伦敦大学学院）；University of Ioannina（约阿尼纳大学）
Many robotic path planning problems are continuous, stochastic, and high-dimensional. The ability of a mobile manipulator to coordinate its base and manipulator in order to control its whole-body online is particularly challenging when self and environment collision avoidance is required. Reinforcement Learning techniques have the potential to solve such problems through their ability to generalise over environments. We study joint penalties and joint limits of a state-of-the-art mobile manipulator whole-body controller that uses LIDAR sensing for obstacle collision avoidance. We propose direc…
Socially-aware Object Transportation by a Mobile Manipulator in Static Planar Environments with Obstacles
Computer Vision and Robotics Laboratory (VeRLab), Department of Computer Science, Universidade Federal de Minas Gerais（计算机视觉与机器人实验室（VeRLab）、计算机科学系、巴西矿务大学）
Existing approaches to direct robot-to-human handovers are typically implemented on fixed-base robot arms, or on mobile manipulators that come to a full stop before performing the handover. We propose "on-the-go" handovers which permit a moving mobile manipulator to hand over an object to a human without stopping. The on-the-go handover motion is generated with a reactive controller that allows simultaneous control of the base and the arm. In a user study, human receivers subjectively assessed on-the-go handovers to be more efficient, predictable, natural, better timed and safer than handovers…
This work addresses the problem of kinematic trajectory planning for mobile manipulators with non-holonomic constraints, and holonomic operational-space tracking constraints. We obtain whole-body trajectories and time-varying kinematic feedback controllers by solving a Constrained Sequential Linear Quadratic Optimal Control problem. The employed algorithm features high efficiency through a continuous-time formulation that benefits from adaptive step-size integrators and through linear complexity in the number of integration steps. In a first application example, we solve kinematic trajectory p…
Mobile manipulator throwing is a promising method to increase the flexibility and efficiency of dynamic manipulation in factories. Its major challenge is to efficiently plan a feasible throw under a wide set of task specifications. We show that the mobile manipulator throwing problem can be simplified to a planar problem, hence greatly reducing the computational costs. Using machine learning approaches, we build a model of the object's inverted flying dynamics and the robot's kinematic feasibility, which enables throwing motion generation within 1 ms for given query of target position. Thanks …
Örebro University（厄勒布鲁大学）；Center for Applied Autonomous Sensor Systems (AASS)（应用自主传感器系统中心）
Integrating the heterogeneous controllers of a complex mechanical system, such as a mobile manipulator, within the same structure and in a modular way is still challenging. In this work we extend our framework based on Behavior Trees for the control of a redundant mechanical system to the problem of commanding more complex systems that involve multiple low-level controllers. This allows the integrated systems to achieve non-trivial goals that require coordination among the sub-systems.
University of Toronto Institute for Aerospace Studies（多伦多大学航空航天研究所）；University of Zagreb（萨格勒布大学）
We present a novel approach for mobile manipulator self-calibration using contact information. Our method, based on point cloud registration, is applied to estimate the extrinsic transform between a fixed vision sensor mounted on a mobile base and an end effector. Beyond sensor calibration, we demonstrate that the method can be extended to include manipulator kinematic model parameters, which involves a non-rigid registration process. Our procedure uses on-board sensing exclusively and does not rely on any external measurement devices, fiducial markers, or calibration rigs. Further, it is full…
Osaka University（大阪大学）；National Institute of Advanced Industrial Science and Technology (AIST)（国立先进工业科学技术研究所（AIST））
In this paper, we present a planner that plans a sequence of base positions for a mobile manipulator to efficiently and robustly collect objects stored in distinct trays. We achieve high efficiency by exploring the common areas where a mobile manipulator can grasp objects stored in multiple trays simultaneously and move the mobile manipulator to the common areas to reduce the time needed for moving the mobile base. We ensure robustness by optimizing the base position with the best clearance to positioning uncertainty so that a mobile manipulator can complete the task even if there is a certain…
University of Modena and Reggio Emilia（摩德纳-雷焦艾米利亚大学）
Human-robot collaborative tasks foresee interactions between humans and robots with various degrees of complexity. Specifically, for tasks which involve physical contact among the agents, challenges arise in the modelling and control of such interaction. In this paper we propose a control architecture capable of ensuring a flexible and robustly stable physical human-robot interaction, focusing on a collaborative transportation task. The architecture is deployed onto a mobile manipulator, modelled as a whole-body structure, which aids the operator during the transportation of an unwieldy load. …
Adaptive falling and recovery skills greatly extend the applicability of robot deployments. In the case of legged mobile manipulators, the robot arm could adaptively stop the fall and assist the recovery. Prior works on falling and recovery strategies for legged mobile manipulators usually rely on assumptions such as inelastic collisions and falling in defined directions to enable real-time computation. This paper presents a learning-based approach to reducing fall damage and recovery. An asymmetric actor-critic training structure is used to train a time-invariant policy with time-varying rewa…
IIIT Hyderabad（国际信息技术研究所海得拉巴分校）；University of Genoa（热那亚大学）；University of Tübingen（图宾根大学）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；IIT Jodhpur（焦特布尔印度理工学院）
In this paper, a kinematic motion planning algorithm for cooperative spatial payload manipulation is presented. A hierarchical approach is introduced to compute real-time collision-free motion plans for a formation of mobile manipulator robots. Initially, collision-free configurations of a deformable 2-D virtual bounding box are identified, over a planning horizon, to define a convex workspace for the entire system. Then, 3-D payload configurations whose projections lie within the defined convex workspace are computed. Finally, a convex decentralized model-predictive controller is formulated t…
Autonomous mobile manipulation offers a dual advantage of mobility provided by a mobile platform and dexterity afforded by the manipulator. In this paper, we present a whole-body optimal control framework to jointly solve the problems of manipulation, balancing and interaction as one optimization problem for an inherently unstable robot. The optimization is performed using a Model Predictive Control (MPC) approach; the optimal control problem is transcribed at the end-effector space, treating the position and orientation tasks in the MPC planner, and skillfully planning for end-effector contac…
HEATS: A Hierarchical Framework for Efficient Autonomous Target Search with Mobile Manipulators
School of Mechanical Engineering and Automation, Harbin Institute of Technology Shenzhen（哈尔滨工业大学（深圳）机械工程与自动化学院）；University of Science and Technology of China（中国科学技术大学）
This paper investigates a fully distributed cooperation scheme for networked mobile manipulators. To achieve cooperative task allocation in a distributed way, an adaptation-based estimation law is established for each robotic agent to estimate the desired local trajectory. In addition, wrench synthesis is analyzed in detail to lay a solid foundation for tight cooperation tasks. Together with the estimated task, a set of distributed adaptive controllers is proposed to achieve motion synchronization of the mobile manipulator ensemble over a directed graph with a spanning tree irrespective of the…
Robotic systems performing end-user oriented autonomous exploration can be deployed in different scenarios which not only require mapping but also simultaneous inspection of regions of interest for the end-user. In this work, we propose a novel Next-Best-View (NBV) planner which can perform full exploration and user-oriented exploration with inspection of the regions of interest using a mobile manipulator robot. We address the exploration-inspection problem as an instance of Multi-Objective Optimization (MOO) and propose a weighted-sum-based information gain function for computing NBVs for the…
Deep reinforcement learning produces robust locomotion policies for legged robots over challenging terrains. To date, few studies have leveraged model-based methods to combine these locomotion skills with the precise control of manipulators. Here, we incorporate external dynamics plans into learning-based locomotion policies for mobile manipulation. We train the base policy by applying a random wrench sequence on the robot base in simulation and adding the noisified wrench sequence prediction to the policy observations. The policy then learns to counteract the partially-known future disturbanc…
Federal University of Minas Gerais（米纳斯吉拉斯联邦大学）；The University of Manchester（曼彻斯特大学）
This paper presents two approaches to obtain the dynamical equations of mobile manipulators using dual quaternion algebra. The first one is based on a general recursive Newton-Euler formulation and uses twists and wrenches, which are propagated through high-level algebraic operations and works for any type of joints and arbitrary parameterizations. The second approach is based on Gauss's Principle of Least Constraint (GPLC) and includes arbitrary equality constraints. In addition to showing the connections of GPLC with Gibbs-Appell and Kane's equations, we use it to model a nonholonomic mobile…
Czech Technical University in Prague（布拉格捷克理工大学）；New York University（纽约大学）；University of Pennsylvania（宾夕法尼亚大学）
Mobile manipulators have the potential to revolutionize modern agriculture, logistics and manufacturing. In this work, we present the design of a ground-based mobile manipulator for automated structure assembly. The proposed system is capable of autonomous localization, grasping, transportation and deployment of construction material in a semi-structured environment. Special effort was put into making the system invariant to lighting changes, and not reliant on external positioning systems. Therefore, the presented system is self-contained and capable of operating in outdoor and indoor conditi…
TopAY: Efficient Trajectory Planning for Differential Drive Mobile Manipulators via Topological Paths Search and Arc Length-Yaw Parameterization
State Key Laboratory of Industrial Control Technology, Zhejiang University, Hangzhou 310027, China（浙江大学工业控制技术状态重点实验室）；Huzhou Institute of Zhejiang University, Huzhou 313000, China（浙江大学湖州研究院）
Italian Institute of Technology（意大利技术研究院）；University of Genoa（热那亚大学）
This paper presents a simplified model-based trajectory optimization (TO) formulation for motion planning on quadruped mobile manipulators that carry heavy payload of known mass. The proposed payload-aware formulation simultaneously plans locomotion, payload manipulation and considers both robot and payload model dynamics while remaining computationally efficient. At the presence of heavy payload, the approach exhibits reduced leg outstretching (thus increased manipulability) in kinematically demanding motions due to the contribution of payload manipulation in the optimization. The framework's…
National Institute of Advanced Industrial Science and Technology (AIST)（国立先进工业科学技术研究所（AIST））；Osaka University（大阪大学）
In this paper, we address efficiently and robustly collecting objects stored in different trays using a mobile manipulator. A resolution complete method, based on precomputed reachability database, is proposed to explore collision-free inverse kinematics (IK) solutions and then a resolution complete set of feasible base positions can be determined. This method approximates a set of representative IK solutions that are especially helpful when solving IK and checking collision are treated separately. For real world applications, we take into account the base positioning uncertainty and plan a se…
Ecole Centrale de Lille（里尔中央理工学院）；Beihang University（北京航空航天大学）
A high redundant non-holonomic humanoid mobile dual-arm manipulator system is presented in this paper where the motion planning to realize "human-like" autonomous navigation and manipulation tasks is studied. Firstly, an improved MaxiMin NSGA-II algorithm, which optimizes five objective functions to solve the problems of singularity, redundancy, and coupling between mobile base and manipulator simultaneously, is proposed to design the optimal pose to manipulate the target object. Then, in order to link the initial pose and that optimal pose, an off-line motion planning algorithm is designed. I…
The COVID-19 pandemic has demonstrated the need for a more effective and efficient disinfection approach to combat infectious diseases. Ultraviolet germicidal irradiation (UVGI) is a proven mean for disinfection and sterilization and has been integrated into handheld devices and autonomous mobile robots. Existing UVGI robots which are commonly equipped with uncovered lamps that emit intense ultraviolet radiation suffer from: inability to be used in human presence, shadowing of objects, and long disinfection time. These robots also have a high operational cost. This paper introduces a cost-effe…
A resolved acceleration control (RAC) and proportional-integral active force control (PIAFC) is proposed as an approach for the robust motion control of a mobile manipulator (MM) comprising a differentially driven wheeled mobile platform with a two-link planar arm mounted on top of the platform. The study emphasizes on the integrated kinematic and dynamic control strategy in which the RAC is used to manipulate the kinematic component while the PIAFC is implemented to compensate the dynamic effects including the bounded known/unknown disturbances and uncertainties. The effectivenss and robustne…
Seconda Università degli Studi di Napoli（那不勒斯第二大学）
A reactive obstacle avoidance method for mobile manipulators is presented. The objectives of the developed algorithm are twofold. The first one is to find a trajectory in the configuration space of a mobile manipulator so as to follow a given trajectory in the task space. The second objective consists in locally adjusting the trajectory in the configuration space in order to avoid collisions with potentially moving obstacles and self-collisions in unstructured and dynamic environments. The perception is exclusively based on a set of proximity sensors distributed on the robot mechanical structu…
National Key Laboratory of Digital Control and System Engineering (DCSELab), Ho Chi Minh City University of Technology (HCMUT)（胡志明市理工大学数字控制与系统工程国家重点实验室）；Ho Chi Minh City University of Technology (HCMUT)（胡志明市理工大学）；Vietnam National University Ho Chi Minh City（越南国立大学胡志明市分校）；Ton Duc Thang University（孙德胜大学）
All flames are extinguished as early as possible, or fire services have to deal with major conflagrations. This leads to the fact that the quality of fire extinguishers has become a very sensitive and important issue in firefighting. Inspired by the development of automatic fire fighting systems, this paper proposes key specifications based on the standard of fire extinguishers that is ISO 7165:2009 and ISO 11601:2008, and feasible solutions to design a mobile manipulator for automatically evaluating the quality or, more specifically, power of fire extinguishers. In addition, a part of the mec…
A Bimanual Gesture Interface for ROS-Based Mobile Manipulators Using TinyML and Sensor Fusion
†‡Department of Mechatronics Engineering, Rajshahi University of Engineering \& Technology, Kazla, Rajshahi-6204, Bangladesh §Department of Electrical \& Systems Engineering, School of Engineering \& Applied Science, University of Pennsylvania, Philadelphia, PA 19104, United States Email: , †, ‡, §
Robust Dynamic Control Barrier Function Based Trajectory Planning for Mobile Manipulator
School of Mechanical Engineering and Automation, Harbin Institute of Technology Shenzhen（哈尔滨工业大学（深圳）机械工程与自动化学院）；Graduated School of Information Science and Technology, Osaka University（大阪大学信息科学与技术研究生院）
Keep it Upright: Model Predictive Control for Nonprehensile Object Transportation with Obstacle Avoidance on a Mobile Manipulator
Technical University of Munich（慕尼黑工业大学）；University of Toronto Institute for Aerospace Studies（多伦多大学航空航天研究所）；University of Toronto Robotics Institute（多伦多大学机器人研究所）；Munich Institute of Robotics and Machine Intelligence（慕尼黑机器人与机器智能研究所）；Vector Institute for Artificial Intelligence（矢量人工智能研究所）
In this letter, we present an interactive probabilistic mapping framework for a mobile manipulator picking objects from a pile. The aim is to map the scene, actively decide where to go next and which object to pick, make changes to the scene by picking the chosen object, and then map these changes alongside. The proposed framework uses a novel dynamic Gaussian Process (GP) Implicit Surface method to incrementally build and update the scene map that reflects environment changes. Actively the framework provides the next-best-view, balancing the need for picking object reachability with map infor…
University of Rochester（罗切斯特大学）；Hajim School of Engineering and Applied Sciences（哈吉姆工程与应用科学学院）
Language is an effective medium for bi-directional communication in human-robot teams. To infer the meaning of many instructions, robots need to construct a model of their surroundings that describe the spatial, semantic, and metric properties of objects from observations and prior information about the environment. Recent algorithms condition the expression of object detectors in a robot's perception pipeline on language to generate a minimal representation of the environment necessary to efficiently determine the meaning of the instruction. We expand on this work by introducing the ability t…
