Anthropomorphism on Risk Perception: The Role of Trust and Domain Knowledge in Decision-Support AI
School of Management and Economics, The Chinese University of Hong Kong, Shenzhen（管理学院和经济学系，香港中文大学（深圳））；Computational Media and Arts, The Hong Kong University of Science and Technology (Guangzhou)（计算媒体与艺术，香港科学与技术大学（广州））
Anthropomorphic design is routinely used to make conversational agents more approachable and engaging. Yet its influence on users' perceptions remains poorly understood. Drawing on psychological theories, we propose that anthropomorphism influences risk perception via two complementary forms of trust, and that domain knowledge moderates these relationships. To test our model, we conducted a large-scale online experiment (N = 1,256) on a financial decision-support system implementing different anthropomorphic designs. We found that anthropomorphism indirectly reduces risk perception by increasing both cognitive and affective trust. Domain knowledge moderates these paths: participants with low financial knowledge experience a negative indirect effect of perceived anthropomorphism on risk perception via cognitive trust, whereas those with high financial knowledge exhibit a positive direct and indirect effect. We discuss theoretical contributions to human-AI interaction and design implications for calibrating trust in anthropomorphic decision-support systems for responsible AI.
Anthropomorphism and Trust in Human-Large Language Model interactions
Brain and Creativity Institute, Dornsife College of Letters, Arts and Sciences, University of Southern California（脑与创造力研究所，文理学院，南加州大学）；USC Mrs. T.H. Chan Division of Occupational Science and Occupational Therapy, University of Southern California（USC 玛丽·T.H.陈职业科学与职业治疗 division，南加州大学）；Psychiatry and Biobehavioral Sciences, David Geffen School of Medicine, University of California, Los Angeles（精神病学与生物行为科学，大卫·格芬医学院，加州大学洛杉矶分校）；Google DeepMind, Zurich（谷歌深Mind，苏黎世）；Google Research（谷歌研究）；Brain and Creativity Institute（脑与创造力研究所）
Recent advances in technology have allowed an automation system to recognize its errors and repair trust more actively than ever. While previous research has called for further studies of different human factors and design features, their effect on human-automation trust repair scenarios remains unknown, especially concerning emotions. This paper seeks to fill such gaps by investigating the impact of anthropomorphism, users' individual differences, and emotional responses on human-automation trust repair. Our experiment manipulated various types of trust violations and apology messages with di…
Modern conversational agents such as Alexa and Google Assistant represent significant progress in speech recognition, natural language processing, and speech synthesis. But as these agents have grown more realistic, concerns have been raised over how their social nature might unconsciously shape our interactions with them. Through a survey of 500 voice assistant users, we explore whether users' relationships with their voice assistants can be quantified using the same metrics as social, interpersonal relationships; as well as if this correlates with how much they trust their devices and the ex…
How human-like do conversational robots need to look to enable long-term human-robot conversation? One essential aspect of long-term interaction is a human's ability to adapt to the varying degrees of a conversational partner's engagement and emotions. Prosodically, this can be achieved through (dis)entrainment. While speech-synthesis has been a limiting factor for many years, restrictions in this regard are increasingly mitigated. These advancements now emphasise the importance of studying the effect of robot embodiment on human entrainment. In this study, we conducted a between-subjects onli…
Örebro University（厄勒布鲁大学）；Robert Bosch GmbH（罗伯特·博世有限公司）；University of Skövde（舍夫德大学）
Robots are increasingly deployed in spaces shared with humans, including home settings and industrial environments. In these environments, the interaction between humans and robots (HRI) is crucial for safety, legibility, and efficiency. A key factor in HRI is trust, which modulates the acceptance of the system. Anthropomorphism has been shown to modulate trust development in a robot, but robots in industrial environments are not usually anthropomorphic. We designed a simple interaction in an industrial environment in which an anthropomorphic mock driver (ARMoD) robot simulates to drive an aut…
Public Opinion and The Rise of Digital Minds: Perceived Risk, Trust, and Regulation Support
Americans for Responsible Innovation（美国负责任创新组织）；Convergence Analysis（汇聚分析机构）；University of Washington（华盛顿大学）；Texas A&M University（得克萨斯农工大学）；Sentience Institute（感知研究所）；Leiden University（莱顿大学）；National Taiwan University（台湾大学）；National Cheng Kung University（成功大学）；University of Chicago（芝加哥大学）；Stanford University（斯坦福大学）
Anthropomorphism in the age of Large Language Models: An overview of potential risks and mitigations
Institute of Intelligent Systems and Robotics（智能系统与机器人研究所）；CNRS（法国国家科学研究中心）；Sorbonne University（索邦大学）；Sciences, Norms, Democracy（科学、规范与民主研究所）；Sorbonne Center for Artificial Intelligence（索邦人工智能中心）；Centre for Research Ethics and Bioethics（研究伦理与生物伦理中心）；Uppsala University（乌普萨拉大学）；Biogem, Biology and Molecular Genetics Research Institute（Biogem生物学与分子遗传学研究所）
Automated decision systems (ADS) have become ubiquitous in many high-stakes domains. Those systems typically involve sophisticated yet opaque artificial intelligence (AI) techniques that seldom allow for full comprehension of their inner workings, particularly for affected individuals. As a result, ADS are prone to deficient oversight and calibration, which can lead to undesirable (e.g., unfair) outcomes. In this work, we conduct an online study with 200 participants to examine people's perceptions of fairness and trustworthiness towards ADS in comparison to a scenario where a human instead of…
Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院（KIT））
Automated decision systems are increasingly used for consequential decision making -- for a variety of reasons. These systems often rely on sophisticated yet opaque models, which do not (or hardly) allow for understanding how or why a given decision was arrived at. This is not only problematic from a legal perspective, but non-transparent systems are also prone to yield undesirable (e.g., unfair) outcomes because their sanity is difficult to assess and calibrate in the first place. In this work, we conduct a study to evaluate different attempts of explaining such systems with respect to their …
Automated decision systems (ADS) are increasingly used for consequential decision-making. These systems often rely on sophisticated yet opaque machine learning models, which do not allow for understanding how a given decision was arrived at. In this work, we conduct a human subject study to assess people's perceptions of informational fairness (i.e., whether people think they are given adequate information on and explanation of the process and its outcomes) and trustworthiness of an underlying ADS when provided with varying types of information about the system. More specifically, we instantia…
"Do I Trust the AI?" Towards Trustworthy AI-Assisted Diagnosis: Understanding User Perception in LLM-Supported Reasoning
School of Information Science and Technology, ShanghaiTech University（上海科技大学信息科学与技术学院）；Shanghai Clinical Research and Trial Center, ShanghaiTech University（上海科技大学上海临床研究试验中心）
Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院）
It is often argued that one goal of explaining automated decision systems (ADS) is to facilitate positive perceptions (e.g., fairness or trustworthiness) of users towards such systems. This viewpoint, however, makes the implicit assumption that a given ADS is fair and trustworthy, to begin with. If the ADS issues unfair outcomes, then one might expect that explanations regarding the system's workings will reveal its shortcomings and, hence, lead to a decrease in fairness perceptions. Consequently, we suggest that it is more meaningful to evaluate explanations against their effectiveness in ena…
Voice assistants offer a convenient and hands-free way of accessing computing in the home, but a key problem with speech as an interaction modality is how to scaffold accurate mental models of voice assistants, a task complicated by privacy and security concerns. We present the results of a survey of voice assistant users (n=1314) measuring trust, security, and privacy perceptions of voice assistants with varying levels of online functionality explained in different ways. We then asked participants to re-explain how these voice assistants worked, showing that while privacy explanations relieve…
What Shapes User Trust in ChatGPT? A Mixed-Methods Study of User Attributes, Trust Dimensions, Task Context, and Societal Perceptions among University Students
The Trusted Caregiver: The Influence of Eye and Mouth Design Incorporating the Baby Schema Effect in Virtual Humanoid Agents on Older Adults Users' Perception of Trustworthiness
Algorithmic processes are increasingly employed to perform managerial decision making, especially after the tremendous success in Artificial Intelligence (AI). This paradigm shift is occurring because these sophisticated AI techniques are guaranteeing the optimality of performance metrics. However, this adoption is currently under scrutiny due to various concerns such as fairness, and how does the fairness of an AI algorithm affects user's trust is much legitimate to pursue. In this regard, we aim to understand the relationship between induced algorithmic fairness and its perception in humans.…
Smart home devices have found their way through people's homes as well as hearts. One such smart device is Amazon Alexa. Amazon Alexa is a voice-controlled application that is rapidly gaining popularity. Alexa was primarily used for checking weather forecasts, playing music, and controlling other devices. This paper tries to explore the extent to which people are aware of the privacy policies pertaining to the Amazon Alexa devices. We have evaluated behavioral change towards their interactions with the device post being aware of the adverse implications. Resulting knowledge will give researche…
Wait or cross? Understanding the influence of behavioral tendency, trust, and risk perception on pedestrian gap-acceptance of automated truck platoons
Faculty of Maritime and Transportation, Ningbo University（宁波大学海运学院）；Collaborative Innovation Center of Modern Urban Traffic Technologies, Southeast University（东南大学现代城市交通技术协同创新中心）；Centre for Transport Engineering and Modelling, Department of Civil and Environmental Engineering, Imperial College London（帝国理工学院土木与环境工程系交通工程与建模中心）；College of Transportation Engineering and the Key Laboratory of Road and Traffic Engineering, Ministry of Education, Tongji University（同济大学交通运输工程学院及教育部道路与交通工程重点实验室）；School of Traffic and Transportation Engineering, Central South University（中南大学交通工程学院）
Risk-limiting audits (RLAs) are expected to strengthen the public confidence in the correctness of an election outcome. We hypothesize that this is not always the case, in part because for large margins between the winner and the runner-up, the number of ballots to be drawn can be so small that voters lose confidence. We conduct a user study with 105 participants resident in the US. Our findings confirm the hypothesis, showing that our study participants felt less confident when they were told the number of ballots audited for RLAs. We elaborate on our findings and propose recommendations for …
Research has shown that human-agent relationships form in similar ways to human-human relationships. Since children do not have the same critical analysis skills as adults (and may over-trust technology, for example), this relationship-formation is concerning. Nonetheless, little research investigates children's perceptions of conversational agents in-depth, and even less investigates how education might change these perceptions. We present K-12 workshops with associated conversational AI concepts to encourage healthier understanding and relationships with agents. Through studies with the curr…
When LLM Rationales Become User-Facing: Effects on Trust Perception, Decision-Making, and Gaze Behaviors
National Institute of Informatics（国立情报学研究所）；University of Amsterdam（阿姆斯特丹大学）；Yale School of Medicine（耶鲁大学医学院）；University of Tokyo（东京大学）；Centrum Wiskunde & Informatica (CWI)（荷兰数学与计算机科学研究中心）；Utrecht University（乌得勒支大学）
