The recognition of actions performed by humans and the anticipation of their intentions are important enablers to yield sociable and successful collaboration in human-robot teams. Meanwhile, robots should have the capacity to deal with multiple objectives and constraints, arising from the collaborative task or the human. In this regard, we propose vision techniques to perform human action recognition and image classification, which are integrated into an Augmented Hierarchical Quadratic Programming (AHQP) scheme to hierarchically optimize the robot's reactive behavior and human ergonomics. The proposed framework allows one to intuitively command the robot in space while a task is being executed. The experiments confirm increased human ergonomics and usability, which are fundamental parameters for reducing musculoskeletal diseases and increasing trust in automation.
Interactive Motion Planning for Human-Robot Collaboration Based on Human-Centric Configuration Space Ergonomic Field
Department of Mechanical and Automation Engineering, T-Stone Robotics Institute, The Chinese University of Hong Kong（机械与自动化工程系，T-Stone机器人研究所，香港中文大学）；Idiap Research Institute（Idiap研究所）；Department of Advanced Robotics, Istituto Italiano di Tecnologia（先进机器人系，意大利Istituto Italiano di Tecnologia）
Integrating Ergonomics and Manipulability for Upper Limb Postural Optimization in Bimanual Human-Robot Collaboration
Department of Mechanical and Automation Engineering, T-Stone Robotics Institute, The Chinese University of Hong Kong（机械与自动化工程系、T-Stone机器人研究所、香港中文大学）；Bristol Robotics Laboratory, UWE Bristol（布里斯托尔机器人实验室、UWE布里斯托尔）；HKCLR, Hong Kong（香港物流机器人中心、香港）
As robotic systems become more and more capable of assisting humans in their everyday lives, we must consider the opportunities for these artificial agents to make their human collaborators feel unsafe or to treat them unfairly. Robots can exhibit antisocial behavior causing physical harm to people or reproduce unfair behavior replicating and even amplifying historical and societal biases which are detrimental to humans they interact with. In this paper, we discuss these issues considering sociable robotic manipulation and fair robotic decision making. We propose a novel approach to learning f…
The high prevalence of work-related musculoskeletal disorders (WMSDs) could be addressed by optimizing Human-Robot Collaboration (HRC) frameworks for manufacturing applications. In this context, this paper proposes two hypotheses for ergonomically effective task delegation and HRC. The first hypothesis states that it is possible to quantify ergonomically professional tasks using motion data from a reduced set of sensors. Then, the most dangerous tasks can be delegated to a collaborative robot. The second hypothesis is that by including gesture recognition and spatial adaptation, the ergonomics…
Istituto Italiano di Tecnologia（意大利理工学院）；The University of Manchester（曼彻斯特大学）
When a human and a humanoid robot collaborate physically, ergonomics is a key factor to consider. Assuming a given humanoid robot, several control architectures exist nowadays to address ergonomic physical human-robot collaboration. This paper takes one step further by considering robot hardware parameters as optimization variables in the problem of collaborative payload lifting. The variables that parametrize robot's kinematics and dynamics ensure their physical consistency, and the human model is considered in the optimization problem. By leveraging the proposed modelling framework, the ergo…
University of Washington（华盛顿大学）；Indian Institute of Technology Gandhinagar（印度理工学院甘地纳加尔分校）
Automated real-time prediction of the ergonomic risks of manipulating objects is a key unsolved challenge in developing effective human-robot collaboration systems for logistics and manufacturing applications. We present a foundational paradigm to address this challenge by formulating the problem as one of action segmentation from RGB-D camera videos. Spatial features are first learned using a deep convolutional model from the video frames, which are then fed sequentially to temporal convolutional networks to semantically segment the frames into a hierarchy of actions, which are either ergonom…
Istituto Italiano di Tecnologia（意大利技术研究院）；University of Genoa（热那亚大学）；Politecnico di Milano（米兰理工大学）
By incorporating ergonomics principles into the task allocation processes, human-robot collaboration (HRC) frameworks can favour the prevention of work-related musculoskeletal disorders (WMSDs). In this context, existing offline methodologies do not account for the variability of human actions and states; therefore, planning and dynamically assigning roles in human-robot teams remains an unaddressed challenge.This study aims to create an ergonomic role allocation framework that optimises the HRC, taking into account task features and human state measurements. The presented framework consists o…
STL-Based Motion Planning and Uncertainty-Aware Risk Analysis for Human-Robot Collaboration with a Multi-Rotor Aerial Vehicle
Department of Cybernetics, Czech Technical University in Prague（捷克技术大学布拉格分校控制学系）；Department of Power Generation Technologies and Materials, Ricerca sul Sistema Energetico S.p.A.（能源系统研究股份有限公司电力生成技术与材料学系）；Robotics and Mechatronics Department, Electrical Engineering, Mathematics, and Computer Science Faculty, University of Twente（代尔夫特理工大学机器人与机电学系，电气工程、数学与计算机科学学院）；Department of Computer, Control and Management Engineering, Sapienza University of Rome（罗马萨皮恩扎大学计算机、控制与管理工程系）
Industrial standards define safety requirements for Human-Robot Collaboration (HRC) in industrial manufacturing. The standards particularly require real-time monitoring and securing of the minimum protective distance between a robot and an operator. In this work, we propose a depth-sensor based model for workspace monitoring and an interactive Augmented Reality (AR) User Interface (UI) for safe HRC. The AR UI is implemented on two different hardware: a projector-mirror setup anda wearable AR gear (HoloLens). We experiment the workspace model and UIs for a realistic diesel motor assembly task. …
Istituto Italiano di Tecnologia（意大利理工学院）；The University of Manchester（曼彻斯特大学）；INAIL（意大利国家工伤事故保险研究所）；Honda R&D Co., Ltd.（本田研发有限公司）
Humans show specialized strategies for efficient collaboration. Transferring similar strategies to humanoid robots can improve their capability to interact with other agents, leading the way to complex collaborative scenarios with multiple agents acting on a shared environment. In this paper we present a control framework for robot-robot collaborative lifting. The proposed shared controller takes into account the joint action of both the robots thanks to a centralized controller that communicates with them, and solves the whole-system optimization. Efficient collaboration is ensured by taking …
In the near future, collaborative robots (cobots) are expected to play a vital role in the manufacturing and automation sectors. It is predicted that workers will work side by side in collaboration with cobots to surpass fully automated factories. In this regard, physical human-robot interaction (pHRI) aims to develop natural communication between the partners to bring speed, flexibility, and ergonomics to the execution of complex manufacturing tasks. One challenge in pHRI is to design an optimal interaction controller to balance the limitations introduced by the contradicting nature of transp…
Adaptive vs. Static Robot-to-Human Handover: A Study on Orientation and Approach Direction
Department of Engineering "Enzo Ferrari", University of Modena and Reggio Emilia（工程学院"Enzo Ferrari"，摩德纳和雷吉奥艾米利亚大学）；Department of Surgery, Medicine, Dentistry and Morphological Sciences, University of Modena and Reggio Emilia（外科、医学、牙科和形态学科学学院，摩德纳和雷吉奥艾米利亚大学）
We present Model-Predictive Interaction Primitives -- a robot learning framework for assistive motion in human-machine collaboration tasks which explicitly accounts for biomechanical impact on the human musculoskeletal system. First, we extend Interaction Primitives to enable predictive biomechanics: the prediction of future biomechanical states of a human partner conditioned on current observations and intended robot control signals. In turn, we leverage this capability within a model-predictive control strategy to identify the future ergonomic and biomechanical ramifications of potential rob…
A Vision Based Framework Integrating Attention and Action Cues for Interpretable Cognitive Workload Assessment in Human-Robot Collaborative Assembly
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；College of Future Technology, The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）未来技术学院）
Towards Shared Embodied Intelligence in Humanoid Robots through Optimization Development and Testing of the Human Aware ergoCub Robot
GenerativeBionics（生成生物技术）；Artificial and Mechanical Intelligence（人工与机械智能）；School of Computer Science, University of Manchester（曼彻斯特大学计算机科学学院）；Humanoid Sensing and Perception（人形感知与感知）；iCub Tech Facility（iCub技术设施）；DiMEILA, Istituto Nazionale Assicurazione Infortuni sul Lavoro (INAIL)（DiMEILA，意大利国家职业伤害保险机构（INAIL））
