Human-robot collaboration (HRC) is an emerging trend of robotics that promotes the co-presence and cooperation of humans and robots in common workspaces. Physical vicinity and interaction between humans and robots, combined with the uncertainty of human behaviour, could lead to undesired situations where humans are injured. Thus, safety is a priority for HRC applications.
Safety analysis via formal modelling and verification techniques could considerably avoid dangerous consequences, but only if the models of HRC systems are comprehensive and realistic, which requires reasonably realistic models of human behaviour. This paper explores state-of-the-art solutions for modelling human and discusses which ones are suitable for HRC scenarios.
ABB Corporate Research（ABB企业研究院）；KTH - Royal Institute of Technology（KTH皇家理工学院）；ABB Robotics（ABB机器人）
In modern industrial collaborative robotic applications, it is desirable to create robot programs automatically, intuitively, and time-efficiently. Moreover, robots need to be controlled by reactive policies to face the unpredictability of the environment they operate in. In this paper we propose a framework that combines a method that learns Behavior Trees (BTs) from demonstration with a method that evolves them with Genetic Programming (GP) for collaborative robotic applications. The main contribution of this paper is to show that by combining the two learning methods we obtain a method that…
University of Modena and Reggio Emilia（摩德纳-雷焦艾米利亚大学）
In collaborative robotic cells, a human operator and a robot share the workspace in order to execute a common job, consisting of a set of tasks. A proper allocation and scheduling of the tasks for the human and for the robot is crucial for achieving an efficient human-robot collaboration. In order to deal with the dynamic and unpredictable behavior of the human and for allowing the human and the robot to negotiate about the tasks to be executed, a two layers architecture for solving the task allocation and scheduling problem is proposed. The first layer optimally solves the task allocation pro…
Safe and efficient collaboration among multiple robots in unstructured environments is increasingly critical in the era of Industry 4.0. However, achieving robust and autonomous collaboration among humans and other robots requires modern robotic systems to have effective proximity perception and reactive obstacle avoidance. In this paper, we propose a novel methodology for reactive whole-body obstacle avoidance that ensures conflict-free robot-robot interactions even in dynamic environment. Unlike existing approaches based on Jacobian-type, sampling based or geometric techniques, our methodolo…
ElasticROS: An Elastically Collaborative Robot Operation System for Fog and Cloud Robotics
Cheng Kar-Shun Robotics Institute, The Hong Kong University of Science and Technology（香港科技大学郑志刚机器人研究院）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）
In this work, we introduce an adaptive control framework for human-robot collaborative transportation of objects with unknown deformation behaviour. The proposed framework takes as input the haptic information transmitted through the object, and the kinematic information of the human body obtained from a motion capture system to create reactive whole-body motions on a mobile collaborative robot. In order to validate our framework experimentally, we compared its performance with an admittance controller during a co-transportation task of a partially deformable object. We additionally demonstrat…
Mitigating Compensatory Movements in Prosthesis Users via Adaptive Collaborative Robotics
Human-Robot Interfaces and Interaction, Istituto Italiano di Tecnologia（人机交互与交互，意大利技术研究院）；Dept. of Informatics, Bioengineering, Robotics and System Engineering, University of Genoa（信息学、生物工程、机器人与系统工程系，热那亚大学）
Model Reconciliation through Explainability and Collaborative Recovery in Assistive Robotics
German Aerospace Center (DLR), Robotics and Mechatronics Center (RMC)（德国航空航天中心（DLR）、机器人与机电中心）；University of Cambridge, Department of Engineering（剑桥大学工程系）；Universität Tübingen, Department of Psychology（图宾根大学心理学系）；Leibniz-Institut für Wissensmedien (IWM)（莱比锡知识媒体研究所）；Technical University of Munich（慕尼黑技术大学）
We present a robotic setup for real-world testing and evaluation of human-robot and human-human collaborative learning. Leveraging the sample-efficiency of the Soft Actor-Critic algorithm, we have implemented a robotic platform able to learn a non-trivial collaborative task with a human partner, without pre-training in simulation, and using only 30 minutes of real-world interactions. This enables us to study Human-Robot and Human-Human collaborative learning through real-world interactions. We present preliminary results, showing that state-of-the-art deep learning methods can take human-robot…
University of Bristol（布里斯托大学）；University of the West of England（西英格兰大学）
Industries such as flexible manufacturing and home care will be transformed by the presence of robotic assistants. Assurance of safety and functional soundness for these robotic systems will require rigorous verification and validation. We propose testing in simulation using Coverage-Driven Verification (CDV) to guide the testing process in an automatic and systematic way. We use a two-tiered test generation approach, where abstract test sequences are computed first and then concretized (e.g., data and variables are instantiated), to reduce the complexity of the test generation problem. To dem…
A*STAR（新加坡科技研究局）；Institute of High Performance Computing, A*STAR（新加坡科技研究局高性能计算研究所）；Institute of Information Research, A*STAR（新加坡科技研究局信息研究所）
Robotic drawing has become increasingly popular as an entertainment and interactive tool. In this paper we present RoboCoDraw, a real-time collaborative robot-based drawing system that draws stylized human face sketches interactively in front of human users, by using the Generative Adversarial Network (GAN)-based style transfer and a Random-Key Genetic Algorithm (RKGA)-based path optimization. The proposed RoboCoDraw system takes a real human face image as input, converts it to a stylized avatar, then draws it with a robotic arm. A core component in this system is the Avatar-GAN proposed by us…
Collaborative human activities are grounded in social and moral norms, which humans consciously and subconsciously use to guide and constrain their decision-making and behavior, thereby strengthening their interactions and preventing emotional and physical harm. This type of norm-based processing is also critical for robots in many human-robot interaction scenarios (e.g., when helping elderly and disabled persons in assisted living facilities, or assisting humans in assembly tasks in factories or even the space station). In this position paper, we will briefly describe how several components i…
University of Coimbra（科英布拉大学）；Arts et Métiers（法国国立高等工程技术学院）
Human-robot collision avoidance is a key in collaborative robotics and in the framework of Industry 4.0. It plays an important role for achieving safety criteria while having humans and machines working side-by-side in unstructured and time-varying environment. This study introduces the subject of manipulator's on-line collision avoidance into a real industrial application implementing typical sensors and a commonly used collaborative industrial manipulator, KUKA iiwa. In the proposed methodology, the human co-worker and the robot are represented by geometric primitives (capsules). The minimum…
Advancements in the domain of physical human-robot interaction (pHRI) have tremendously improved the ability of humans and robots to communicate, collaborate, and coexist. In particular, compliant robotic systems offer many characteristics that can be leveraged towards enabling physical interactions that more efficiently and intuitively communicate intent, making compliant systems potentially useful in more physically demanding subsets of human-robot collaborative scenarios. Tensegrity robots are an example of compliant systems that are well-suited to physical interactions while still retainin…
Skolkovo Institute of Science and Technology (Skoltech)（斯科尔科沃科学技术研究所（Skoltech））
The projection mapping systems on the human face is limited by the latency and the movement of the users. The area of the projection is restricted by the position of the projectors and the cameras. We are introducing MaskBot, a real-time projection mapping system operated by a 6 Degrees of Freedom (DoF) collaborative robot. The collaborative robot locates the projector and camera in normal position to the face of the user to increase the projection area and to reduce the latency of the system. A webcam is used to detect the face and to sense the robot-user distance to modify the projection siz…
In this paper we propose FlexHRC+, a hierarchical human-robot cooperation architecture designed to provide collaborative robots with an extended degree of autonomy when supporting human operators in high-variability shop-floor tasks. The architecture encompasses three levels, namely for perception, representation, and action. Building up on previous work, here we focus on (i) an in-the-loop decision making process for the operations of collaborative robots coping with the variability of actions carried out by human operators, and (ii) the representation level, integrating a hierarchical AND/OR…
Establishing common ground between an intelligent robot and a human requires communication of the robot's intention, behavior, and knowledge to the human to build trust and assure safety in a shared environment. This paper introduces SENSAR (Seeing Everything iN Situ with Augmented Reality), an augmented reality robotic system that enables robots to communicate their sensory and cognitive data in context over the real-world with rendered graphics, allowing a user to understand, correct, and validate the robot's perception of the world. Our system aims to support human-robot interaction researc…
University of Genova（热那亚大学）；Italian Institute of Technology（意大利理工学院）
As robotic technology advances, the barriers to the coexistence of humans and robots are slowly coming down. Application domains like elderly care, collaborative manufacturing, collaborative manipulation, etc., are considered the need of the hour, and progress in robotics holds the potential to address many societal challenges. The future socio-technical systems constitute of blended workforce with a symbiotic relationship between human and robot partners working collaboratively. This thesis attempts to address some of the research challenges in enabling human-robot collaboration. In particula…
This paper presents a system integration approach for a 6-DoF (Degree of Freedom) collaborative robot to operate a pipette for liquid dispensing. Its technical development is threefold. First, we designed an end-effector for holding and triggering manual pipettes. Second, we took advantage of a collaborative robot to recognize labware poses and planned robotic motion based on the recognized poses. Third, we developed vision-based classifiers to predict and correct positioning errors and thus precisely attached pipettes to disposable tips. Through experiments and analysis, we confirmed that the…
University of Bristol（布里斯托大学）；University of the West of England（西英格兰大学）
The challenges of robotic software testing extend beyond conventional software testing. Valid, realistic and interesting tests need to be generated for multiple programs and hardware running concurrently, deployed into dynamic environments with people. We investigate the use of Belief-Desire-Intention (BDI) agents as models for test generation, in the domain of human-robot interaction (HRI) in simulations. These models provide rational agency, causality, and a reasoning mechanism for planning, which emulate both intelligent and adaptive robots, as well as smart testing environments directed by…
Cellular-enabled Collaborative Robots Planning and Operations for Search-and-Rescue Scenarios
i2CAT Foundation（i2CAT基金会）；NEC Laboratories Europe GmbH（欧洲NEC实验室）；Polytechnic University of Catalonia, UPC（加泰罗尼亚理工大学（UPC））；Institut Català de Recerca i Estudis Avançats, ICREA（加泰罗尼亚高级研究所（ICREA））
Institute of Electronics and Informatics Engineering of Aveiro, University of Aveiro（阿威罗大学阿威罗电子与信息工程研究院）；University of Aveiro（阿威罗大学）；Computer Vision Center（计算机视觉中心）；Escuela Superior Politécnica del Litoral（海岸高等理工学院）
Collaborative robotic industrial cells are workspaces where robots collaborate with human operators. In this context, safety is paramount, and for that a complete perception of the space where the collaborative robot is inserted is necessary. To ensure this, collaborative cells are equipped with a large set of sensors of multiple modalities, covering the entire work volume. However, the fusion of information from all these sensors requires an accurate extrinsic calibration. The calibration of such complex systems is challenging, due to the number of sensors and modalities, and also due to the …
Jožef Stefan Institute（约泽夫·斯蒂芬研究所）；Jožef Stefan International Postgraduate School（约泽夫·斯蒂芬国际研究生院）
An important factor in developing control models for human-robot collaboration is how acceptable they are to their human partners. One such method for creating acceptable control models is to attempt to mimic human-like behaviour in robots so that their actions appear more intuitive to humans. To investigate how task complexity affects human perception and acceptance of their robot partner, we propose a novel human-based robot control model for obstacle avoidance that can account for the leader-follower dynamics that normally occur in human collaboration. The performance and acceptance of the …
For collaborative robots to become useful, end users who are not robotics experts must be able to instruct them to perform a variety of tasks. With this goal in mind, we developed a system for end-user creation of robust task plans with a broad range of capabilities. CoSTAR: the Collaborative System for Task Automation and Recognition is our winning entry in the 2016 KUKA Innovation Award competition at the Hannover Messe trade show, which this year focused on Flexible Manufacturing. CoSTAR is unique in how it creates natural abstractions that use perception to represent the world in a way use…
There are many situations in which it would be beneficial for a robot to have predictive abilities similar to those of rational humans. Some of these situations include collaborative robots, robots in adversarial situations, and for dynamic obstacle avoidance. This paper presents an approach to modeling behaviors of dynamic agents in order to empower robots with the ability to predict the agent's actions and identify the behavior the agent is executing in real time. The method of behavior modeling implemented uses hidden Markov models (HMMs) to model the unobservable states of the dynamic agen…
Istituto Italiano di Tecnologia（意大利技术研究院）；Politecnico di Milano（米兰理工大学）
Human-robot collaborative assembly systems enhance the efficiency and productivity of the workplace but may increase the workers' cognitive demand. This paper proposes an online and quantitative framework to assess the cognitive workload induced by the interaction with a co-worker, either a human operator or an industrial collaborative robot with different control strategies. The approach monitors the operator's attention distribution and upper-body kinematics benefiting from the input images of a low-cost stereo camera and cutting-edge artificial intelligence algorithms (i.e. head pose estima…
This paper presents an experimental adaptation of a non-collaborative robot arm to collaborate with the environment, as one step towards adapting legacy robotic machinery to fit in industry 4.0 requirements. A cloud-based internet of things (CIoT) service is employed to connect, supervise and control a robotic arm's motion using the added wireless sensing devices to the environment. A programmable automation controller (PAC) unit, connected to the robot arm receives the most recent changes and updates the motion of the robot arm. The experimental results show that the proposed non-expensive se…
In three laboratory experiments, we examine the impact of personally relevant failures (PeRFs) on perceptions of a collaborative robot. PeR is determined by how much a specific issue applies to a particular person, i.e., it affects one's own goals and values. We hypothesized that PeRFs would reduce trust in the robot and the robot's Likeability and Willingness to Use (LWtU) more than failures that are not personal to participants. To achieve PeR in human-robot interaction, we utilized three different manipulation mechanisms: A) damage to property, B) financial loss, and C) first-person versus …
Osaka University（大阪大学）；University of California, Santa Cruz（加州大学圣克鲁兹分校）
This paper develops a robotic manipulation planner for human-robot collaborative assembly. Unlike previous methods which study an independent and fully AI-equipped autonomous system, this paper explores the subtask distribution between a robot and a human and studies a human-in-the-loop robotic system for collaborative assembly. The system distributes the subtasks of an assembly to robots and humans by exploiting their advantages and avoiding their disadvantages. The robot in the system will work on pick-and-place tasks and provide workpieces to humans. The human collaborator will work on fine…
Digital Twin technology is playing a pivotal role in the modern industrial evolution. Especially, with the technological progress in the Internet-of-Things (IoT) and the increasing trend in autonomy, multi-sensor equipped robotics can create practical digital twin, which is particularly useful in the industrial applications for operations, maintenance and safety. Herein, we demonstrate a real-world digital twin of a safety-critical robotics applications with a Franka-Emika-Panda robotic arm. We develop and showcase an edge-assisted collaborative digital twin for dynamic obstacle avoidance whic…
