Explain yourself! Effects of Explanations in Human-Robot Interaction
Universität Hamburg（汉堡大学）
Jakob Ambsdorf, Alina Munir, Yiyao Wei, Klaas Degkwitz, Harm Matthias Harms, Susanne Stannek, Kyra Ahrens, Dennis Becker, Erik Strahl, Tom Weber, Stefan Wermter
备注：
Accepted at 2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)
Recent developments in explainable artificial intelligence promise the potential to transform human-robot interaction: Explanations of robot decisions could affect user perceptions, justify their reliability, and increase trust. However, the effects on human perceptions of robots that explain their decisions have not been studied thoroughly. To analyze the effect of explainable robots, we conduct a study in which two simulated robots play a competitive board game. While one robot explains its moves, the other robot only announces them. Providing explanations for its actions was not sufficient to change the perceived competence, intelligence, likeability or safety ratings of the robot. However, the results show that the robot that explains its moves is perceived as more lively and human-like. This study demonstrates the need for and potential of explainable human-robot interaction and the wider assessment of its effects as a novel research direction.
Enhancing Robot Explanation Capabilities through Vision-Language Models: a Preliminary Study by Interpreting Visual Inputs for Improved Human-Robot Interaction
This work documents a pilot user study evaluating the effectiveness of contrastive, causal and example explanations in supporting human understanding of AI in a hypothetical commonplace human robot interaction HRI scenario. In doing so, this work situates explainable AI XAI in the context of the social sciences and suggests that HRI explanations are improved when informed by the social sciences.
Despite great advances in what robots can do, they still experience failures in human-robot collaborative tasks due to high randomness in unstructured human environments. Moreover, a human's unfamiliarity with a robot and its abilities can cause such failures to repeat. This makes the ability to failure explanation very important for a robot. In this work, we describe a user study that incorporated different robotic failures in a human-robot collaboration (HRC) task aimed at filling a shelf. We included different types of failures and repeated occurrences of such failures in a prolonged intera…
Explainable artificial intelligence (XAI) can help foster trust in and acceptance of intelligent and autonomous systems. Moreover, understanding the motivation for an agent's behavior results in better and more successful collaborations between robots and humans. However, not only can humans benefit from a robot's explanation but the robot itself can also benefit from explanations given to him. Currently, most attention is paid to explaining deep neural networks and black-box models. However, a lot of these approaches are not applicable to humanoid robots. Therefore, in this position paper, cu…
Most research on human-robot handovers focuses on the development of comfortable and efficient HRI; few have studied handover failures. If a failure occurs in the beginning of the interaction, it prevents the whole handover process and destroys trust. Here we analyze the underlying reasons why people want explanations in a handover scenario where a robot cannot possess the object. Results suggest that participants set expectations on their request and that a robot should provide explanations rather than non-verbal cues after failing. Participants also expect that their handover request can be …
Personalised Explanations in Long-term Human-Robot Interactions
PAL robotics (Barcelona, Spain)（PAL机器人公司（巴塞罗那，西班牙））；Institut de Robòtica i Informàtica Industrial (IRI-CSIC)（机器人与信息技术研究所（IRI-CSIC））；Universitat Politècnica de Catalunya - BarcelonaTech (UPC)（巴塞罗那技术大学（UPC））；University of Hamburg（汉堡大学）；Artificial Intelligence Research Institute (IIIA-CSIC)（人工智能研究所（IIIA-CSIC））
Franklin & Marshall College（富兰克林与马歇尔学院）；Massachusetts Institute of Technology（麻省理工学院）；Occidental College（西方学院）
There is a need for socially assistive robots (SARs) to provide transparency in their behavior by explaining their reasoning. Additionally, the reasoning and explanation should represent the user's preferences and goals. To work towards satisfying this need for interpretable reasoning and representations, we propose the robot uses Analogical Theory of Mind to infer what the user is trying to do and uses the Hint Engine to find an appropriate assistance based on what the user is trying to do. If the user is unsure or confused, the robot provides the user with an explanation, generated by the Ex…
Center for Vision, Cognition, Learning, and Autonomy, UCLA（加州大学洛杉矶分校视觉、认知、学习与自主中心）；Massachusetts Institute of Technology（麻省理工学院）
Human collaborators can effectively communicate with their partners to finish a common task by inferring each other's mental states (e.g., goals, beliefs, and desires). Such mind-aware communication minimizes the discrepancy among collaborators' mental states, and is crucial to the success in human ad-hoc teaming. We believe that robots collaborating with human users should demonstrate similar pedagogic behavior. Thus, in this paper, we propose a novel explainable AI (XAI) framework for achieving human-like communication in human-robot collaborations, where the robot builds a hierarchical mind…
Dialogue based Interactive Explanations for Safety Decisions in Human Robot Collaboration
Department of Civil Engineering and Management, Faculty of Science and Engineering, The University of Manchester（曼彻斯特大学科学与工程学院土木工程与管理系）；VRAIN, Universitat Politècnica de València（瓦伦西亚理工大学VRAIN研究所）；Department of Engineering, University of Cambridge（剑桥大学工程系）；Department of Mechanical and Aerospace Engineering, Faculty of Science and Engineering, The University of Manchester（曼彻斯特大学科学与工程学院机械与航空航天工程系）；Center for the Possible Futures, Kyoto Institute of Technology（京都工艺纤维大学未来可能性中心）；Institute of Industrial Science, The University of Tokyo（东京大学生产技术研究所）；Graduate School of Frontier Sciences, The University of Tokyo（东京大学新领域创成科学研究科）
Explainable AI techniques that describe agent reward functions can enhance human-robot collaboration in a variety of settings. One context where human understanding of agent reward functions is particularly beneficial is in the value alignment setting. In the value alignment context, an agent aims to infer a human's reward function through interaction so that it can assist the human with their tasks. If the human can understand where gaps exist in the agent's reward understanding, they will be able to teach more efficiently and effectively, leading to quicker human-agent team performance impro…
While thinking aloud has been reported to positively affect problem-solving, the effects of the presence of an embodied entity (e.g., a social robot) to whom words can be directed remain mostly unexplored. In this work, we investigated the role of a robot in a "rubber duck debugging" setting, by analyzing how a robot's listening behaviors could support a thinking-aloud problem-solving session. Participants completed two different tasks while speaking their thoughts aloud to either a robot or an inanimate object (a giant rubber duck). We implemented and tested two types of listener behavior in …
Automated vehicles (AVs) are social robots that can potentially benefit our society. According to the existing literature, AV explanations can promote passengers' trust by reducing the uncertainty associated with the AV's reasoning and actions. However, the literature on AV explanations and trust has failed to consider how the type of trust - cognitive versus affective - might alter this relationship. Yet, the existing literature has shown that the implications associated with trust vary widely depending on whether it is cognitive or affective. To address this shortcoming and better understand…
Aesthetics of Robot-Mediated Applied Drama: A Case Study on REMind
Systems Design Engineering, University of Waterloo（滑铁卢大学系统设计工程系）；Electrical & Computer Engineering, University of Waterloo（滑铁卢大学电气与计算机工程系）；Communication Arts, University of Waterloo（滑铁卢大学传播艺术系）
Colorado School of Mines（科罗拉多矿业学院）；George Mason University（乔治梅森大学）；University of Massachusetts Lowell（马萨诸塞大学洛厄尔分校）
Autonomous robots must communicate about their decisions to gain trust and acceptance. When doing so, robots must determine which actions are causal, i.e., which directly give rise to the desired outcome, so that these actions can be included in explanations. In behavior learning in psychology, this sort of reasoning during an action sequence has been studied extensively in the context of imitation learning. And yet, these techniques and empirical insights are rarely applied to human-robot interaction (HRI). In this work, we discuss the relevance of behavior learning insights for robot intent …
Carnegie Mellon University（卡内基梅隆大学）；INESC-ID；Instituto Superior Técnico, Universidade de Lisboa（里斯本大学高等技术学院）；Universidade de Lisboa（里斯本大学）；Instituto Universitário de Lisboa (ISCTE-IUL)（里斯本大学学院（ISCTE-IUL））
Social robots are becoming increasingly diverse in their design, behavior, and usage. In this chapter, we provide a broad-ranging overview of the main characteristics that arise when one considers social robots and their interactions with humans. We specifically contribute a framework for characterizing social robots along 7 dimensions that we found to be most relevant to their design. These dimensions are: appearance, social capabilities, purpose and application area, relational role, autonomy and intelligence, proximity, and temporal profile. Within each dimension, we account for the variety…
Model Reconciliation through Explainability and Collaborative Recovery in Assistive Robotics
German Aerospace Center (DLR), Robotics and Mechatronics Center (RMC)（德国航空航天中心（DLR）、机器人与机电中心）；University of Cambridge, Department of Engineering（剑桥大学工程系）；Universität Tübingen, Department of Psychology（图宾根大学心理学系）；Leibniz-Institut für Wissensmedien (IWM)（莱比锡知识媒体研究所）；Technical University of Munich（慕尼黑技术大学）
Teachy Mini: Development and Preliminary Evaluation of a Knowledge-Based Generative Social Robot for Higher Education
Institute of Information Systems, Zurich University of Applied Sciences（苏黎世应用科学大学信息系统研究所）；Center for Cognitive Interaction Technology, Bielefeld University（比勒费尔德大学认知交互技术中心）
Massachusetts Institute of Technology（麻省理工学院）；Bar Ilan University（巴伊兰大学）；University of Sussex（苏塞克斯大学）；University College London（伦敦大学学院）
To interact seamlessly with robots, users must infer the causes of a robot's behavior and be confident about that inference. Hence, trust is a necessary condition for human-robot collaboration (HRC). Despite its crucial role, it is largely unknown how trust emerges, develops, and supports human interactions with nonhuman artefacts. Here, we review the literature on trust, human-robot interaction, human-robot collaboration, and human interaction at large. Early models of trust suggest that trust entails a trade-off between benevolence and competence, while studies of human-to-human interaction …
Norwegian University of Science and Technology (NTNU)（挪威科技大学 (NTNU)）
We investigate the effect of including domain knowledge about a robotic system's causal relations when generating explanations. To this end, we compare two methods from explainable artificial intelligence, the popular KernelSHAP and the recent causal SHAP, on a deep neural network trained using deep reinforcement learning on the task of controlling a lever using a robotic manipulator. A primary disadvantage of KernelSHAP is that its explanations represent only the features' direct effects on a model's output, not considering the indirect effects a feature can have on the output by affecting ot…
University of Massachusetts Lowell（马萨诸塞大学洛厄尔分校）；New England Robotics Validation and Experimentation Center（新英格兰机器人验证与实验中心）；NERVE Center（NERVE中心）
As robot systems become more ubiquitous, developing understandable robot systems becomes increasingly important in order to build trust. In this paper, we present an approach to developing a holistic robot explanation system, which consists of three interconnected components: state summarization, storage and querying, and human interface. To find trends towards and gaps in the development of such an integrated system, a literature review was performed and categorized around those three components, with a focus on robotics applications. After the review of each component, we discuss our propose…
Ulm University（乌尔姆大学）；University of Sarajevo（萨拉热窝大学）
The continued development of robots has enabled their wider usage in human surroundings. Robots are more trusted to make increasingly important decisions with potentially critical outcomes. Therefore, it is essential to consider the ethical principles under which robots operate. In this paper we examine how contrastive and non-contrastive explanations can be used in understanding the ethics of robot action plans. We build upon an existing ethical framework to allow users to make suggestions about plans and receive automatically generated contrastive explanations. Results of a user study indica…
Norwegian University of Science and Technology（挪威科技大学）；University of Melbourne（墨尔本大学）
Although many machine learning methods, especially from the field of deep learning, have been instrumental in addressing challenges within robotic applications, we cannot take full advantage of such methods before these can provide performance and safety guarantees. The lack of trust that impedes the use of these methods mainly stems from a lack of human understanding of what exactly machine learning models have learned, and how robust their behaviour is. This is the problem the field of explainable artificial intelligence aims to solve. Based on insights from the social sciences, we know that…
One challenge with introducing robots into novel environments is misalignment between supervisor expectations and reality, which can greatly affect a user's trust and continued use of the robot. We performed an experiment to test whether the presence of an explanation of expected robot behavior affected a supervisor's trust in an autonomous robot. We measured trust both subjectively through surveys and objectively through a dual-task experiment design to capture supervisors' neglect tolerance (i.e., their willingness to perform their own task while the robot is acting autonomously). Our object…
Robot navigation with deep reinforcement learning (RL) achieves higher performance and performs well under complex environment. Meanwhile, the interpretation of the decision-making of deep RL models becomes a critical problem for more safety and reliability of autonomous robots. In this paper, we propose a visual explanation method based on an attention branch for deep RL models. We connect attention branch with pre-trained deep RL model and the attention branch is trained by using the selected action by the trained deep RL model as a correct label in a supervised learning manner. Because the …
