Extended Abstract Accepted and Presented at The Communication Challenges in Joint Action for Human-Robot Interaction Workshop, International Conference on Social Robotics (ICSR) 2019, Madrid, Spain
Robots existed as separate entities till now, but the horizons of a symbiotic human-robot partnership are impending. Despite all the recent technical advances in terms of hardware, robots are still not endowed with desirable relational skills that ensure a social component in their existence. This article draws from our experience as roboticists in Human-Robot Collaboration (HRC) with humanoid robots and presents some of the recent advances made towards realizing intuitive robot behaviors and partner-aware control involving physical interactions.
Robot facial expressions and gaze are important factors for enhancing human-robot interaction (HRI), but their effects on human collaboration and perception are not well understood, for instance, in collaborative game scenarios. In this study, we designed a collaborative triadic HRI game scenario, where two participants worked together to insert objects into a shape sorter. One participant assumed the role of a guide. The guide instructed the other participant, who played the role of an actor, by placing occluded objects into the sorter. A humanoid robot issued instructions, observed the inter…
With the advances in robotic technology, research in human-robot collaboration (HRC) has gained in importance. For robots to interact with humans autonomously they need active decision making that takes human partners into account. However, state-of-the-art research in HRC does often assume a leader-follower division, in which one agent leads the interaction. We believe that this is caused by the lack of a reliable representation of the human and the environment to allow autonomous decision making. This problem can be overcome by an embodied approach to HRC which is inspired by psychological s…
Institute for Systems and Robotics, Instituto Superior Técnico（系统与机器人研究所，里斯本高等理工学院）
Collaborative space robots are an emerging technology with high impact as robots facilitate servicing functions in collaboration with astronauts with higher precision during lengthy tasks, under tight operational schedules, with less risk and costs, making them more efficient and economically more viable. However, human-robot collaboration in space is still a challenge concerning key issues in human-robot interaction, including mobility and collaborative manipulation of objects on a microgravity environment. In this paper we formulate an algorithm that enables a free-flyer robot, equipped with…
Carnegie Mellon University（卡内基梅隆大学）；University of California, Berkeley（加州大学伯克利分校）
Human motion prediction is non-trivial in modern industrial settings. Accurate prediction of human motion can not only improve efficiency in human robot collaboration, but also enhance human safety in close proximity to robots. Among existing prediction models, the parameterization and identification methods of those models vary. It remains unclear what is the necessary parameterization of a prediction model, whether online adaptation of the model is necessary, and whether prediction can help improve safety and efficiency during human robot collaboration. These problems result from the difficu…
Robotic Vision for Human-Robot Interaction and Collaboration: A Survey and Systematic Review
Australian Research Council Centre of Excellence for Robotic Vision（澳大利亚研究理事会机器人视觉卓越中心）；Queensland University of Technology（昆士兰科技大学）；Monash University（莫纳什大学）；University of Oxford（牛津大学）
University of Modena and Reggio Emilia（摩德纳-雷焦艾米利亚大学）
A fruitful collaboration is based on the mutual knowledge of each other skills and on the possibility of communicating their own limits and proposing alternatives to adapt the execution of a task to the capabilities of the collaborators. This paper aims at reproducing such a scenario in a human-robot collaboration setting by proposing a novel communication control architecture. Exploiting control barrier functions, the robot is made aware of its (dynamic) skills and limits and, thanks to a local predictor, it is able to assess if it is possible to execute a requested task and, if not, to propo…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Carnegie Mellon University（卡内基梅隆大学）
A robot needs multiple interaction modes to robustly collaborate with a human in complicated industrial tasks. We develop a Coexistence-and-Cooperation (CoCo) human-robot collaboration system. Coexistence mode enables the robot to work with the human on different sub-tasks independently in a shared space. Cooperation mode enables the robot to follow human guidance and recover failures. A human intention tracking algorithm takes in both human and robot motion measurements as input and provides a switch on the interaction modes. We demonstrate the effectiveness of CoCo system in a use case analo…
Humans engaged in collaborative activities are naturally able to convey their intentions to teammates through multi-modal communication, which is made up of explicit and implicit cues. Similarly, a more natural form of human-robot collaboration may be achieved by enabling robots to convey their intentions to human teammates via multiple communication channels. In this paper, we postulate that a better communication may take place should collaborative robots be able to anticipate their movements to human teammates in an intuitive way. In order to support such a claim, we propose a robot system'…
Human collaborators coordinate effectively their actions through both verbal and non-verbal communication. We believe that the the same should hold for human-robot teams. We propose a formalism that enables a robot to decide optimally between doing a task and issuing an utterance. We focus on two types of utterances: verbal commands, where the robot expresses how it wants its human teammate to behave, and state-conveying actions, where the robot explains why it is behaving this way. Human subject experiments show that enabling the robot to issue verbal commands is the most effective form of co…
Stochastic Decision-Making Framework for Human-Robot Collaboration in Industrial Applications
Department of Control and Instrumentation Engineering, King Fahd University of Petroleum and Minerals (KFUPM)（控制与仪器工程系，国王法赫德石油与矿物大学）；Interdisciplinary Research Center for Intelligent Manufacturing & Robotics, King Fahd University of Petroleum and Minerals (KFUPM)（智能制造与机器人交叉研究中心，国王法赫德石油与矿物大学）
Robots are envisioned to work alongside humans in applications ranging from in-home assistance to collaborative manufacturing. Research on human-robot collaboration (HRC) has helped develop various aspects of social intelligence necessary for robots to participate in effective, fluid collaborations with humans. However, HRC research has focused on dyadic, structured, and minimal collaborations between humans and robots that may not fully represent the large scale and emergent nature of more complex, unstructured collaborative activities. Thus, there remains a need for shared testbeds, datasets…
IDAGC: Adaptive Generalized Human-Robot Collaboration via Human Intent Estimation and Multimodal Policy Learning
Chinese Academy of Sciences（中国科学院）；Institute of Automation（自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Zhejiang University（浙江大学）；Beijing Zhongke Huiling Robot Technology Co., LTD（北京中科创联机器人科技有限公司）
Istituto Italiano di Tecnologia（意大利技术研究院）；Politecnico di Milano（米兰理工大学）
The recognition of actions performed by humans and the anticipation of their intentions are important enablers to yield sociable and successful collaboration in human-robot teams. Meanwhile, robots should have the capacity to deal with multiple objectives and constraints, arising from the collaborative task or the human. In this regard, we propose vision techniques to perform human action recognition and image classification, which are integrated into an Augmented Hierarchical Quadratic Programming (AHQP) scheme to hierarchically optimize the robot's reactive behavior and human ergonomics. The…
University of Modena and Reggio Emilia（摩德纳-雷焦艾米利亚大学）
During physical human robot collaboration, it is important to be able to implement a time-varying interactive behaviour while ensuring robust stability. Admittance control and passivity theory can be exploited for achieving these objectives. Nevertheless, when the admittance dynamics is time-varying, it can happen that, for ensuring a passive and stable behaviour, some spurious dissipative effects have to be introduced in the admittance dynamics. These effects are perceived by the user and degrade the collaborative performance. In this paper we exploit the task redundancy of the manipulator in…
University of California, Berkeley（加州大学伯克利分校）；Carnegie Mellon University（卡内基梅隆大学）
Human robot collaboration (HRC) is becoming increasingly important as the paradigm of manufacturing is shifting from mass production to mass customization. The introduction of HRC can significantly improve the flexibility and intelligence of automation. However, due to the stochastic and time-varying nature of human collaborators, it is challenging for the robot to efficiently and accurately identify the plan of human and respond in a safe manner. To address this challenge, we propose an integrated human robot collaboration framework in this paper which includes both plan recognition and traje…
Tsinghua University（清华大学）；Hong Kong Productivity Council（香港生产力促进局）
There is invariably a trade-off between safety and efficiency for collaborative robots (cobots) in human-robot collaborations. Robots that interact minimally with humans can work with high speed and accuracy but cannot adapt to new tasks or respond to unforeseen changes, whereas robots that work closely with humans can but only by becoming passive to humans, meaning that their main tasks suspended and efficiency compromised. Accordingly, this paper proposes a new complementary framework for human-robot collaboration that balances the safety of humans and the efficiency of robots. In this frame…
Research on human-robot collaboration or human-robot teaming, has focused predominantly on understanding and enabling collaboration between a single robot and a single human. Extending human-robot collaboration research beyond the dyad, raises novel questions about how a robot should distribute resources among group members and about what the social and task related consequences of the distribution are. Methodological advances are needed to allow researchers to collect data about human robot collaboration that involves multiple people. This paper presents Tower Construction, a novel resource d…
To enable a natural and fluent human robot collaboration flow, it is critical for a robot to comprehend their human peers' on-going actions, predict their behaviors in the near future, and plan its actions correspondingly. Specifically, the capability of making early predictions is important, so that the robot can foresee the precise timing of a turn-taking event and start motion planning and execution early enough to smooth the turn-taking transition. Such proactive behavior would reduce human's waiting time, increase efficiency and enhance naturalness in collaborative task. To that end, this…
Close human-robot interaction (HRI), especially in industrial scenarios, has been vastly investigated for the advantages of combining human and robot skills. For an effective HRI, the validity of currently available human-machine communication media or tools should be questioned, and new communication modalities should be explored. This article proposes a modular architecture allowing human operators to interact with robots through different modalities. In particular, we implemented the architecture to handle gestural and touchscreen input, respectively, using a smartwatch and a tablet. Finall…
Exploring Human-Robot Collaboration: Analysis of Interaction Modalities in Challenging Tasks
Dalle Molle Institute for Artificial Intelligence (IDSIA), USI-SUPSI（达勒莫勒人工智能研究所（IDSIA））；Department of Surgery, Medicine, Dentistry and Morphological Sciences with interest in Transplant, Oncology and Regenerative Medicine, University of Modena and Reggio Emilia（外科、医学、牙科和形态学科学系（关注移植、肿瘤学和再生医学），摩德纳和雷吉奥艾米利亚大学）；Department of Sciences and Methods for Engineering (DISMI), University of Modena and Reggio Emilia（工程科学与方法系（DISMI），摩德纳和雷吉奥艾米利亚大学）
In human-robot teams, humans often start with an inaccurate model of the robot capabilities. As they interact with the robot, they infer the robot's capabilities and partially adapt to the robot, i.e., they might change their actions based on the observed outcomes and the robot's actions, without replicating the robot's policy. We present a game-theoretic model of human partial adaptation to the robot, where the human responds to the robot's actions by maximizing a reward function that changes stochastically over time, capturing the evolution of their expectations of the robot's capabilities. …
Human-robot collaboration has significant potential in recycling due to the wide variation in the composition of recyclable products. Six participants performed a recyclable item sorting task collaborating with a robot arm equipped with a vision system. The effect of three different levels of human involvement or assistance to the robot (Level 1- occlusion removal; Level 2- optimal spacing; Level 3- optimal grip) on performance metrics such as robot accuracy, task time and subjective fluency were assessed. Results showed that human involvement had a remarkable impact on the robot's accuracy, w…
We address the problem of adapting robot trajectories to improve safety, comfort, and efficiency in human-robot collaborative tasks. To this end, we propose CoMOTO, a trajectory optimization framework that utilizes stochastic motion prediction models to anticipate the human's motion and adapt the robot's joint trajectory accordingly. We design a multi-objective cost function that simultaneously optimizes for i) separation distance, ii) visibility of the end-effector, iii) legibility, iv) efficiency, and v) smoothness. We evaluate CoMOTO against three existing methods for robot trajectory gener…
Integrating Ergonomics and Manipulability for Upper Limb Postural Optimization in Bimanual Human-Robot Collaboration
Department of Mechanical and Automation Engineering, T-Stone Robotics Institute, The Chinese University of Hong Kong（机械与自动化工程系、T-Stone机器人研究所、香港中文大学）；Bristol Robotics Laboratory, UWE Bristol（布里斯托尔机器人实验室、UWE布里斯托尔）；HKCLR, Hong Kong（香港物流机器人中心、香港）
It is well-known that a deep understanding of co-workers' behavior and preference is important for collaboration effectiveness. In this work, we present a method to accomplish smooth human-robot collaboration in close proximity by taking into account the human's behavior while planning the robot's trajectory. In particular, we first use an occupancy map to summarize human's movement preference over time, and such prior information is then considered in an optimization-based motion planner via two cost items as introduced in [1]: 1) avoidance of the workspace previously occupied by human, to el…
This paper presents a planning algorithm designed to improve cooperative robot behavior concerning human comfort during forceful human-robot physical interaction. Particularly, we are interested in planning for object grasping and positioning ensuring not only stability against the exerted human force but also empowering the robot with capabilities to address and improve human experience and comfort. Herein, comfort is addressed as both the muscular activation level required to exert the cooperative task, and the human spatial perception during the interaction, namely, the peripersonal space. …
Human-robot collaboration has gained a notable prominence in Industry 4.0, as the use of collaborative robots increases efficiency and productivity in the automation process. However, it is necessary to consider the use of mechanisms that increase security in these environments, as the literature reports that risk situations may exist in the context of human-robot collaboration. One of the strategies that can be adopted is the visual recognition of the collaboration environment using machine learning techniques, which can automatically identify what is happening in the scene and what may happe…
Peking University（北京大学）；Carnegie Mellon University（卡内基梅隆大学）；Georgia Institute of Technology（佐治亚理工学院）
To engender safe and efficient human-robot collaboration, it is critical to generate high-fidelity predictions of human behavior. The challenges in making accurate predictions lie in the stochasticity and heterogeneity in human behaviors. This paper introduces a method for human trajectory and intention prediction through a multi-task model that is adaptable across different human subjects. We develop a nonlinear recursive least square parameter adaptation algorithm (NRLS-PAA) to achieve online adaptation. The effectiveness and flexibility of the proposed method has been validated in experimen…
