In order to enable physical human-robot interaction where humans and (mobile) manipulators share their workspace and work together, robots have to be equipped with important capabilities to guarantee human safety. The robots have to recognize possible collisions with the human co-worker and react anticipatorily by adapting their motion to avert dangerous situations while they are executing their task.
Therefore, methods have been developed that allow to monitor the workspace of mobile manipulators using multiple depth sensors to gather information about the robot environment. This encompasses both 3D information about obstacles in the close robot surroundings and the prediction of obstacle motions in the entire monitored space. Based on this information, a collision-free robot motion is planned and during the execution the robot continuously reacts to unforeseen dangerous situations by adapting its planned motion, slowing down or stopping.
For the demonstration of a manufacturing scenario, the developed methods have been implemented on a prototypical mobile manipulator. The algorithms handle both robot platform and manipulator in a uniform manner so that an overall optimization of the path and of the collision avoidance behavior is possible. By integrating the monitoring, planning, and interaction control components, the task of grasping, placing and delivering objects to humans in a shared workspace is demonstrated.
Robotic platforms are highly programmable, scalable and versatile to complete several tasks including Inspection, Maintenance and Repair (IMR). Mobile robotics offer reduced restrictions in operating environments, resulting in greater flexibility; operation at height, dangerous areas and repetitive tasks. Cyber physical infrastructures have been identified by the UK Robotics Growth Partnership as a key enabler in how we utilize and interact with sensors and machines via the virtual and physical worlds. Cyber Physical Systems (CPS) allow for robotics and artificial intelligence to adapt and rep…
Recent work in robotic manipulation focuses on object retrieval in cluttered spaces under occlusion. Nevertheless, the majority of efforts lack an analysis of conditions for the completeness of the approaches or the methods apply only when objects can be removed from the workspace. This work formulates the general, occlusion-aware manipulation task, and focuses on safe object reconstruction in a confined space with in-place rearrangement. It proposes a framework that ensures safety with completeness guarantees. Furthermore, an algorithm, which is an instantiation of this abstract framework for…
University of Pennsylvania（宾夕法尼亚大学）；Washington University in St. Louis（圣路易斯华盛顿大学）
In this paper we address mobile manipulation planning problems in the presence of sensing and environmental uncertainty. In particular, we consider mobile sensing manipulators operating in environments with unknown geometry and uncertain movable objects, while being responsible for accomplishing tasks requiring grasping and releasing objects in a logical fashion. Existing algorithms either do not scale well or neglect sensing and/or environmental uncertainty. To face these challenges, we propose a hybrid control architecture, where a symbolic controller generates high-level manipulation comman…
Korea Institute of Science and Technology（韩国科学技术研究院）
We present an algorithm determining where to relocate objects inside a cluttered and confined space while rearranging objects to retrieve a target object. Although methods that decide what to remove have been proposed, planning for the placement of removed objects inside a workspace has not received much attention. Rather, removed objects are often placed outside the workspace, which incurs additional laborious work (e.g., motion planning and execution of the manipulator and the mobile base, perception of other areas). Some other methods manipulate objects only inside the workspace but without…
Catch It! Learning to Catch in Flight with Mobile Dexterous Hands
Shanghai Qi Zhi Institute（上海期智研究院）；Tsinghua University（清华大学）；Shanghai AI Lab（上海人工智能实验室）；Georgia Institute of Technology（佐治亚理工学院）；Stanford University（斯坦福大学）
Bielefeld University of Applied Sciences（比勒费尔德应用科学大学）；Otto-von-Guericke University Magdeburg（奥托-冯-格里克马格德堡大学）
We address the problem of interactively controlling the workspace of a mobile robot to ensure a human-aware navigation. This is especially of relevance for non-expert users living in human-robot shared spaces, e.g. home environments, since they want to keep the control of their mobile robots, such as vacuum cleaning or companion robots. Therefore, we introduce virtual borders that are respected by a robot while performing its tasks. For this purpose, we employ a RGB-D Google Tango tablet as human-robot interface in combination with an augmented reality application to flexibly define virtual bo…
Bielefeld University of Applied Sciences（比勒费尔德应用科学大学）；Otto-von-Guericke University Magdeburg（马格德堡奥托·冯·格里克大学）
Virtual borders are employed to allow humans the interactive and flexible restriction of their mobile robots' workspaces in human-centered environments, e.g. to exclude privacy zones from the workspace or to indicate certain areas for working. They have been successfully specified in interaction processes using methods from human-robot interaction. However, these methods often lack an expressive feedback system, are restricted to robot's on-board interaction capabilities and require a direct line of sight between human and robot. This negatively affects the user experience and interaction time…
University of Applied Sciences Bielefeld（比勒费尔德应用科学大学）；Otto-von-Guericke University Magdeburg（马格德堡奥托·冯·格里克大学）
The increasing number of robots in home environments leads to an emerging coexistence between humans and robots. Robots undertake common tasks and support the residents in their everyday life. People appreciate the presence of robots in their environment as long as they keep the control over them. One important aspect is the control of a robot's workspace. Therefore, we introduce virtual borders to precisely and flexibly define the workspace of mobile robots. First, we propose a novel framework that allows a person to interactively restrict a mobile robot's workspace. To show the validity of t…
This work proposes a novel transformation termed the conformal navigation transformation to achieve collision-free navigation of a robot in a workspace populated with arbitrary polygonal obstacles. The properties of the conformal navigation transformation in the polygonal workspace are investigated in this work as well as its capability to provide a solution to the navigation problem. %The properties of the conformal navigation transformation are investigated, which contribute to the solution of the robot navigation problem in complex polygonal environments. %which facilitates the navigation o…
In this paper, an analysis of the sensing volume coverage of robot workspace as well as the shared human-robot collaborative workspace for various configurations of on-robot Time-of-Flight (ToF) sensor array rings is presented. A methodology for volumetry using octrees to quantify the detection/sensing volume of the sensors is proposed. The change in sensing volume coverage by increasing the number of sensors per ToF sensor array ring and also increasing the number of rings mounted on robot link is also studied. Considerations of maximum ideal volume around the robot workspace that a given ToF…
Belarusian State University of Informatics and Radioelectronics（白俄罗斯信息与无线电电子大学）；Institut de Recherche en Communications et Cybernétique de Nantes（南特通信与控制研究所）
The paper addresses kinematic and geometrical aspects of the Orthoglide, a three-DOF parallel mechanism. This machine consists of three fixed linear joints, which are mounted orthogonally, three identical legs and a mobile platform, which moves in the Cartesian x-y-z space with fixed orientation. New solutions to solve inverse/direct kinematics are proposed and we perform a detailed workspace and singularity analysis, taking into account specific joint limit constraints.
Industrial standards define safety requirements for Human-Robot Collaboration (HRC) in industrial manufacturing. The standards particularly require real-time monitoring and securing of the minimum protective distance between a robot and an operator. In this work, we propose a depth-sensor based model for workspace monitoring and an interactive Augmented Reality (AR) User Interface (UI) for safe HRC. The AR UI is implemented on two different hardware: a projector-mirror setup anda wearable AR gear (HoloLens). We experiment the workspace model and UIs for a realistic diesel motor assembly task. …
Robots are widely deployed in space environments because of their versatility and robustness. However, adverse gravity conditions and challenging terrain geometry expose the limitations of traditional robot designs, which are often forced to sacrifice one of mobility or manipulation capabilities to attain the other. Prospective climbing operations in these environments reveals a need for small, compact robots capable of versatile mobility and manipulation. We propose a novel robotic concept called ReachBot that fills this need by combining two existing technologies: extendable booms and mobile…
IIIT Hyderabad（国际信息技术研究所海得拉巴分校）；University of Genoa（热那亚大学）；University of Tübingen（图宾根大学）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；IIT Jodhpur（焦特布尔印度理工学院）
In this paper, a kinematic motion planning algorithm for cooperative spatial payload manipulation is presented. A hierarchical approach is introduced to compute real-time collision-free motion plans for a formation of mobile manipulator robots. Initially, collision-free configurations of a deformable 2-D virtual bounding box are identified, over a planning horizon, to define a convex workspace for the entire system. Then, 3-D payload configurations whose projections lie within the defined convex workspace are computed. Finally, a convex decentralized model-predictive controller is formulated t…
Laboratoire des Sciences du Numérique de Nantes（南特数字科学实验室）；École Polytéchnique de Montréal（蒙特利尔理工学院）
This paper presents the kinematic analysis of the 3-PPPS parallel robot with an equilateral mobile platform and a U-shape base. The proposed design and appropriate selection of parameters allow to formulate simpler direct and inverse kinematics for the manipulator under study. The parallel singularities associated with the manipulator depend only on the orientation of the end-effector, and thus depend only on the orientation of the end effector. The quaternion parameters are used to represent the aspects, i.e. the singularity free regions of the workspace. A cylindrical algebraic decomposition…
Massachusetts Institute of Technology（麻省理工学院）；Pennsylvania State University（宾夕法尼亚州立大学）；University of California at Santa Barbara（加州大学圣巴巴拉分校）
The most widely applied strategy for workload sharing is to equalize the workload assigned to each resource. In mobile multi-agent systems, this principle directly leads to equitable partitioning policies in which (i) the workspace is divided into subregions of equal measure, (ii) there is a bijective correspondence between agents and subregions, and (iii) each agent is responsible for service requests originating within its own subregion. In this paper, we design provably correct, spatially-distributed and adaptive policies that allow a team of agents to achieve a convex and equitable partiti…
Fast and Safe Trajectory Optimization for Mobile Manipulators With Neural Configuration Space Distance Field
Robotics and Autonomous Systems Thrust, The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学机器人与自主系统研究组）；Department of Electrical and Computer Engineering, National University of Singapore（新加坡国立大学电子与计算机工程系）；Idiap Research Institute（Idiap研究所）；Institute of Data Science, The University of Hong Kong（香港大学数据科学研究所）
Bielefeld University of Appl. Sciences（比勒费尔德应用科学大学）；Otto-von-Guericke University Magdeburg（马格德堡奥托-冯-格里克大学）
We address the problem of controlling the workspace of a 3-DoF mobile robot. In a human-robot shared space, robots should navigate in a human-acceptable way according to the users' demands. For this purpose, we employ virtual borders, that are non-physical borders, to allow a user the restriction of the robot's workspace. To this end, we propose an interaction method based on a laser pointer to intuitively define virtual borders. This interaction method uses a previously developed framework based on robot guidance to change the robot's navigational behavior. Furthermore, we extend this framewo…
Chalmers University of Technology（查尔姆斯理工大学）；Lund University（隆德大学）
Motion planning methods like navigation functions and harmonic potential fields provide (almost) global convergence and are suitable for obstacle avoidance in dynamically changing environments due to their reactive nature. A common assumption in the control design is that the robot operates in a disjoint star world, i.e. all obstacles are strictly starshaped and mutually disjoint. However, in real-life scenarios obstacles may intersect due to expanded obstacle regions corresponding to robot radius or safety margins. To broaden the applicability of aforementioned reactive motion planning method…
Czech Technical University in Prague（布拉格捷克技术大学）；Faculty of Electrical Engineering, Czech Technical University in Prague（布拉格捷克技术大学电气工程学院）
The need to guarantee safety of collaborative robots limits their performance, in particular, their speed and hence cycle time. The standard ISO/TS 15066 defines the Power and Force Limiting operation mode and prescribes force thresholds that a moving robot is allowed to exert on human body parts during impact, along with a simple formula to obtain maximum allowed speed of the robot in the whole workspace. In this work, we measure the forces exerted by two collaborative manipulators (UR10e and KUKA LBR iiwa) moving downward against an impact measuring device. First, we empirically show that th…
Soft robots are promising for manipulation tasks thanks to their compliance, safety, and high degree of freedom. However, the commonly used bidirectional continuum segment design means soft robotic manipulators only function in a limited hemispherical workspace. This work increases a soft robotic arm's workspace by designing, fabricating, and controlling an additional soft prismatic actuator at the base of the soft arm. This actuator consists of pneumatic artificial muscles and a piston, making the actuator back-driveable. We increase the task space volume by 116\%, and we are now able to perf…
Brown University（布朗大学）；Mitsubishi Electric Research Labs（三菱电机研究实验室）
Dynamic movement primitives are widely used for learning skills which can be demonstrated to a robot by a skilled human or controller. While their generalization capabilities and simple formulation make them very appealing to use, they possess no strong guarantees to satisfy operational safety constraints for a task. In this paper, we present constrained dynamic movement primitives (CDMP) which can allow for constraint satisfaction in the robot workspace. We present a formulation of a non-linear optimization to perturb the DMP forcing weights regressed by locally-weighted regression to admit a…
Saint Petersburg State University（圣彼得堡国立大学）；University of New South Wales（新南威尔士大学）
A single nonholonomic under-actuated mobile robot with a bounded control range travels in a 3D workspace. This workspace also hosts a compact unknown domain with a smooth boundary. The robot has access to the horizontal distance to this domain and to a certain direction (typically, vertical) in the space, along with its coordinate (typically, altitude) in this direction. We present a new navigation law that drives the robot to the desired distance to the domain and ensures subsequent full scan of its surface within a desired range of "altitudes". The proposed strategy is computationally inexpe…
Robot programming methods for industrial robots are time consuming and often require operators to have knowledge in robotics and programming. To reduce costs associated with reprogramming, various interfaces using augmented reality have recently been proposed to provide users with more intuitive means of controlling robots in real-time and programming them without having to code. However, most solutions require the operator to be close to the real robot's workspace which implies either removing it from the production line or shutting down the whole production line due to safety hazards. We pro…
University of Oslo（奥斯陆大学）；Oslo University Hospital（奥斯陆大学医院）
With advancing technologies, robotic manipulators and visual environment sensors are becoming cheaper and more widespread. However, robot control can be still a limiting factor for better adaptation of these technologies. Robotic manipulators are performing very well in structured workspaces, but do not adapt well to unexpected changes, like people entering the workspace. We present a method combining 3D Camera based workspace mapping, and a predictive and reflexive robot manipulator trajectory estimation to allow more efficient and safer operation in dynamic workspaces. In experiments on a re…
University of Southern California（南加州大学）；National Institute of Standards and Technology（美国国家标准与技术研究院）
Mobile manipulators can be used for machine tending and material handling tasks in small volume manufacturing applications. These applications usually have semi-structured work environment. The use of a fully autonomous mobile manipulator for such applications can be risky, as an inaccurate model of the workspace may result in damage to expensive equipment. On the other hand, the use of a fully teleoperated mobile manipulator may require a significant amount of operator time. In this paper, a semi-autonomous mobile manipulator is developed for safely and efficiently carrying out machine tendin…
Physical Human-Robot Interaction (pHRI) task involves tight coupling between safety constraints and compliance with human intentions. In this paper, a novel switched model reference admittance controller is developed to maintain compliance with the external force while upholding safety constraints in the workspace for an n-link manipulator involved in pHRI. A switched reference model is designed for the admittance controller to generate the reference trajectory within the safe workspace. The stability analysis of the switched reference model is carried out by an appropriate selection of the Co…
We present a motion planning algorithm to compute collision-free and smooth trajectories for high-DOF robots interacting with humans in a shared workspace. Our approach uses offline learning of human actions along with temporal coherence to predict the human actions. Our intention-aware online planning algorithm uses the learned database to compute a reliable trajectory based on the predicted actions. We represent the predicted human motion using a Gaussian distribution and compute tight upper bounds on collision probabilities for safe motion planning. We also describe novel techniques to acco…
Northeastern University（东北大学）；Ocean University of China（中国海洋大学）
Navigation functions provide both path and motion planning, which can be used to ensure obstacle avoidance and convergence in the sphere world. When dealing with complex and realistic scenarios, constructing a transformation to the sphere world is essential and, at the same time, challenging. This work proposes a novel transformation termed the conformal navigation transformation to achieve collision-free navigation of a robot in a workspace populated with obstacles of arbitrary shapes. The properties of the conformal navigation transformation, including uniqueness, invariance of navigation pr…
In this paper, we develop estimation and control methods for quickly reacting to collisions between omnidirectional mobile platforms and their environment. To enable the full-body detection of external forces, we use torque sensors located in the robot's drivetrain. Using model based techniques we estimate, with good precision, the location, direction, and magnitude of collision forces, and we develop an admittance controller that achieves a low effective mass in reaction to them. For experimental testing, we use a facility containing a calibrated collision dummy and our holonomic mobile platf…
We present a novel class of robots belonging to Constrained Collaborative Mobile Agents (CCMA) family which consists of ground mobile bases with non-holonomic constraints. Moreover, these mobile robots are constrained by closed-loop kinematic chains consisting of revolute joints which can be either passive or actuated. We also describe a novel trajectory optimization method which is general with respect to number of mobile robots, topology of the closed-loop kinematic chains and placement of the actuators at the revolute joints. We also extend the standalone trajectory optimization method to o…
Max-Planck-Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Worcester Polytechnic Institute（伍斯特理工学院）
To enable safe and efficient human-robot collaboration in shared workspaces it is important for the robot to predict how a human will move when performing a task. While predicting human motion for tasks not known a priori is very challenging, we argue that single-arm reaching motions for known tasks in collaborative settings (which are especially relevant for manufacturing) are indeed predictable. Two hypotheses underlie our approach for predicting such motions: First, that the trajectory the human performs is optimal with respect to an unknown cost function, and second, that human adaptation …
The University of Tokyo（东京大学）；Kitasato University Hospital（北里大学医院）
The limited workspace in pediatric endoscopic surgery makes surgical suturing one of the most difficult tasks. During suturing, surgeons have to prevent collisions between tools and also collisions with the surrounding tissues. Surgical robots have been shown to be effective in adult laparoscopy, but assistance for suturing in constrained workspaces has not been yet fully explored. In this letter, we propose guidance virtual fixtures to enhance the performance and the safety of suturing while generating the required task constraints using constrained optimization and Cartesian force feedback. …
