Communication in both human-human and human-robot interac-tion (HRI) contexts consists of verbal (speech-based) and non-verbal(facial expressions, eye gaze, gesture, body pose, etc.) components.The verbal component contains semantic and affective information;accordingly, HRI work on the gesture component so far has focusedon rule-based (mapping words to gestures) and data-driven (deep-learning) approaches to generating speech-paired gestures basedon either semantics or the affective state. Consequently, most ges-ture systems are confined to producing either semantically-linkedor affect-based gesticures. This paper introduces an approach forenabling human-robot communication based on a theory-drivenapproach to generate speech-paired robot gestures using both se-mantic and affective information. Our model takes as input textand sentiment analysis, and generates robot gestures in terms oftheir shape, intensity, and speed.
Georgia Tech Center for Music Technology（佐治亚理工学院音乐技术中心）
As human-robot collaboration opportunities continue to expand, trust becomes ever more important for full engagement and utilization of robots. Affective trust, built on emotional relationship and interpersonal bonds is particularly critical as it is more resilient to mistakes and increases the willingness to collaborate. In this paper we present a novel model built on music-driven emotional prosody and gestures that encourages the perception of a robotic identity, designed to avoid uncanny valley. Symbolic musical phrases were generated and tagged with emotional information by human musicians…
Touch and Tell: Multimodal Decoding of Human Emotions and Social Gestures for Robots
Faculty of Engineering and Architecture（工程与建筑学院）；IDLab-AIRO, Ghent University – imec（IDLab-AIRO，根特大学–imec）；Department of Engineering Science, University of Oxford（工程科学系，牛津大学）
FABG : End-to-end Imitation Learning for Embodied Affective Human-Robot Interaction
Institute of Machine Intelligence, University of Shanghai for Science and Technology（上海理工大学机器智能研究院）；Shanghai Droid Robot Co., Ltd.（上海智舱机器人有限公司）；University of Hamburg（汉堡大学）
Automated vehicles (AVs) are social robots that can potentially benefit our society. According to the existing literature, AV explanations can promote passengers' trust by reducing the uncertainty associated with the AV's reasoning and actions. However, the literature on AV explanations and trust has failed to consider how the type of trust - cognitive versus affective - might alter this relationship. Yet, the existing literature has shown that the implications associated with trust vary widely depending on whether it is cognitive or affective. To address this shortcoming and better understand…
Université de Sherbrooke（舍布鲁克大学）；Defence Research and Development Canada（加拿大国防研究与发展部）
We propose a novel low-complexity lidar gesture recognition system for mobile robot control robust to gesture variation. Our system uses a modular approach, consisting of a pose estimation module and a gesture classifier. Pose estimates are predicted from lidar scans using a Convolutional Neural Network trained using an existing stereo-based pose estimation system. Gesture classification is accomplished using a Long Short-Term Memory network and uses a sequence of estimated body poses as input to predict a gesture. Breaking down the pipeline into two modules reduces the dimensionality of the i…
Heriot-Watt University（赫瑞-瓦特大学）；University of Glasgow（格拉斯哥大学）
We present an ongoing project that aims to develop a social robot to help children cope with painful and distressing medical procedures in a clinical setting. Our approach uses automated planning as a core component for action selection in order to generate plans that include physical, sensory, and social actions for the robot to use when interacting with humans. A key capability of our system is that the robot's behaviour adapts based on the affective state of the child patient. The robot must operate in a challenging physical and social environment where appropriate and safe interaction with…
Anthropomorphic robot avatars present a conceptually novel approach to remote affective communication, allowing people across the world a wider specter of emotional and social exchanges over traditional 2D and 3D image data. However, there are several limitations of current telepresence robots, such as the high weight, complexity of the system that prevents its fast deployment, and the limited workspace of the avatars mounted on either static or wheeled mobile platforms. In this paper, we present a novel concept of telecommunication through a robot avatar based on an anthropomorphic swarm of d…
Text-writing robots have been used in assistive writing and drawing applications. However, robots do not convey emotional tones in the writing process due to the lack of behaviors humans typically adopt. To examine how people interpret designed robotic expressions of emotion through both movements and textual output, we used a pen-plotting robot to generate texts by performing human-like behaviors like stop-and-go, speed, and pressure variation. We examined how people convey emotion in the writing process by observing how they wrote in different emotional contexts. We then mapped these human e…
In the Arms of a Robot: Designing Autonomous Hugging Robots with Intra-Hug Gestures
Max Planck Institute for Intelligent Systems（智能系统马克斯·普朗克研究所）；University of Copenhagen Copenhagen Denmark（哥本哈根大学哥本哈根丹麦）；Max Planck Institute for Intelligent Systems Stuttgart Germany（智能系统马克斯·普朗克研究所斯图加特德国）；University of Copenhagen（哥本哈根大学）
ESATIC (École Supérieure Africaine des Technologies de l’Information et de la Communication)（非洲高等信息与通信技术学院）；LASTIC（LASTIC实验室）；Institut Mines Télécom Atlantique（大西洋矿业电信学院）；U2IS ENSTA Paris（巴黎ENSTA大学U2IS实验室）；Institut Polytechnique de Paris（巴黎理工学院）；INRIA（法国国家信息与自动化研究所）
With the outlook of improving communication and social abilities of people with ASD, we propose to extend the paradigm of robot-based imitation games to ASD teenagers. In this paper, we present an interaction scenario adapted to ASD teenagers, propose a computational architecture using the latest machine learning algorithm Openpose for human pose detection, and present the results of our basic testing of the scenario with human caregivers. These results are preliminary due to the number of session (1) and participants (4). They include a technical assessment of the performance of Openpose, as …
HAPI: A Model for Learning Robot Facial Expressions from Human Preferences
Graduate School of Informatics, Kyoto University（京都大学信息学研究生院）；Psychological Process Research Team, Guardian Robot Project, RIKEN（守护机器人项目心理过程研究组，RIKEN）；Field Science Education and Research Center, Kyoto University（京都大学田野科学教育研究中心）；Interactive Robot Research Team, Guardian Robot Project, RIKEN（守护机器人项目交互机器人研究组，RIKEN）；Research and Development Center for Large Language Models, National Institute of Informatics（信息处理研究所大型语言模型研究开发中心）
The Chinese University of Hong Kong（香港中文大学）；The Hong Kong Polytechnic University（香港理工大学）
Automated surgical gesture recognition is of great importance in robot-assisted minimally invasive surgery. However, existing methods assume that training and testing data are from the same domain, which suffers from severe performance degradation when a domain gap exists, such as the simulator and real robot. In this paper, we propose a novel unsupervised domain adaptation framework which can simultaneously transfer multi-modality knowledge, i.e., both kinematic and visual data, from simulator to real robot. It remedies the domain gap with enhanced transferable features by using temporal cues…
RoboGesture: Real-Time Semantic-aligned Co-Speech Gestures Generation for Humanoid Interaction
Tsinghua University（清华大学）；Galbot Inc.（Galbot公司）；Beijing Institute of Technology（北京理工大学）；Harbin Institute of Technology（哈尔滨工业大学）；Peking University（北京大学）；Shanghai Qi Zhi Institute（上海智知研究院）
Despite significant developments in the design of surgical robots and automated techniques for objective evaluation of surgical skills, there are still challenges in ensuring safety in robot-assisted minimally-invasive surgery (RMIS). This paper presents a runtime monitoring system for the detection of executional errors during surgical tasks through the analysis of kinematic data. The proposed system incorporates dual Siamese neural networks and knowledge of surgical context, including surgical tasks and gestures, their distributional similarities, and common error modes, to learn the differe…
