A virtual embodiment can benefit conversational agents, but it is unclear how their personalities and non-verbal behavior influence the User Experience and Social Presence in Augmented Reality (AR). We asked 30 users to converse with a virtual assistant who gives recommendations about city activities. The participants interacted with two different personalities: Sammy, a cheerful blue mouse, and Olive, a serious green human-like agent. Each was presented with two body languages - happy/friendly and annoyed/unfriendly. We conclude how agent representation and humor affect User Experience aspects, and that body language is significant in the evaluation and perception of the AR agent.
We present the first systematic analysis of personality dimensions developed specifically to describe the personality of speech-based conversational agents. Following the psycholexical approach from psychology, we first report on a new multi-method approach to collect potentially descriptive adjectives from 1) a free description task in an online survey (228 unique descriptors), 2) an interaction task in the lab (176 unique descriptors), and 3) a text analysis of 30,000 online reviews of conversational agents (Alexa, Google Assistant, Cortana) (383 unique descriptors). We aggregate the results…
From Fixed to Flexible: Shaping AI Personality in Context-Sensitive Interaction
The University of Sydney（悉尼大学）；Nanjing University of Information Science and Technology（南京信息工程大学）；University of New South Wales（新南威尔士大学）；University of Melbourne（墨尔本大学）
Exploring the Impact of Personality Traits on Conversational Recommender Systems: A Simulation with Large Language Models
Chinese University of Hong Kong（香港中文大学）；National University of Singapore（新加坡国立大学）；Singapore Management University（新加坡管理大学）；Hong Kong Baptist University（香港浸会大学）；Huazhong University of Science and Technology（华中科技大学）
Autonomous driving research currently faces data sparsity in representation of risky scenarios. Such data is both difficult to obtain ethically in the real world, and unreliable to obtain via simulation. Recent advances in virtual reality (VR) driving simulators lower barriers to tackling this problem in simulation. We propose the first data collection framework for risky scenario driving data from real humans using VR, as well as accompanying numerical driving personality characterizations. We validate the resulting dataset with statistical analyses and model driving behavior with an eight-fa…
There is a growing recognition that artists use valuable ways to understand and work with cognitive and perceptual mechanisms to convey desired experiences and narrative in their created artworks (DiPaola et al., 2010; Zeki, 2001). This paper documents our attempt to computationally model the creative process of a portrait painter, who relies on understanding human traits (i.e., personality and emotions) to inform their art. Our system includes an empathic conversational interaction component to capture the dominant personality category of the user and a generative AI Portraiture system that u…
Conversational recommender systems (CRSs) imitate human advisors to assist users in finding items through conversations and have recently gained increasing attention in domains such as media and e-commerce. Like in human communication, building trust in human-agent communication is essential given its significant influence on user behavior. However, inspiring user trust in CRSs with a "one-size-fits-all" design is difficult, as individual users may have their own expectations for conversational interactions (e.g., who, user or system, takes the initiative), which are potentially related to the…
We describe experiments towards building a conversational digital assistant that considers the preferred conversational style of the user. In particular, these experiments are designed to measure whether users prefer and trust an assistant whose conversational style matches their own. To this end we conducted a user study where subjects interacted with a digital assistant that responded in a way that either matched their conversational style, or did not. Using self-reported personality attributes and subjects' feedback on the interactions, we built models that can reliably predict a user's pre…
South China University of Technology（华南理工大学）；UBTECH Robotics Corp（优必选科技股份有限公司）；Tsinghua University（清华大学）
Human language expression is based on the subjective construal of the situation instead of the objective truth conditions, which means that speakers' personalities and emotions after cognitive processing have an important influence on conversation. However, most existing datasets for conversational AI ignore human personalities and emotions, or only consider part of them. It's difficult for dialogue systems to understand speakers' personalities and emotions although large-scale pre-training language models have been widely used. In order to consider both personalities and emotions in the proce…
Elder-Sim: A Psychometrically Validated Platform for Personality-Stable Elderly Digital Twins
Yulin AI-Enhanced HealthCare Lab（云林人工智能增强医疗实验室）；School of Nursing, Fudan University（复旦大学护理学院）；School of Nursing, Medical College of Soochow University（苏州大学医学院护理学院）；School of Nursing, Dali University（大理大学护理学院）；Department of Advanced Interdisciplinary Studies, The University of Tokyo（东京大学先进跨学科研究部）；The First Affiliated Hospital of Soochow University（苏州大学第一附属医院）；Department of Nursing, The First Affiliated Hospital, Jiangxi Medical College, Nanchang University（江西医学院护理部，南昌大学第一附属医院）；School of Social Development, East China University of Political Science and Law（中国政法大学社会发展学院）；Rory Meyers College of Nursing, New York University（纽约大学 Rory Meyers 护理学院）；New York University Shanghai（纽约大学上海校区）
With the advancements in social robotics and virtual avatars, it becomes increasingly important that these agents adapt their behavior to the mood, feelings and personality of their users. One such aspect of the user is empathy. Whereas many studies measure empathy through offline measures that are collected after empathic stimulation (e.g. post-hoc questionnaires), the current study aimed to measure empathy online, using brain activity collected during the experience. Participants watched an affective 360 video of a child experiencing domestic violence in a virtual reality headset while their…
We present a dialogue elicitation study to assess how users envision conversations with a perfect voice assistant (VA). In an online survey, N=205 participants were prompted with everyday scenarios, and wrote the lines of both user and VA in dialogues that they imagined as perfect. We analysed the dialogues with text analytics and qualitative analysis, including number of words and turns, social aspects of conversation, implied VA capabilities, and the influence of user personality. The majority envisioned dialogues with a VA that is interactive and not purely functional; it is smart, proactiv…
Interaction design for Augmented Reality (AR) is gaining increasing attention from both academia and industry. This survey discusses 260 articles (68.8% of articles published between 2015 - 2019) to review the field of human interaction in connected cities with emphasis on augmented reality-driven interaction. We provide an overview of Human-City Interaction and related technological approaches, followed by a review of the latest trends of information visualization, constrained interfaces, and embodied interaction for AR headsets. We highlight under-explored issues in interface design and inpu…
CoGen3D: An Agentic Human-AI Co-Design Pipeline for 3D Asset Generation for Virtual Reality
Nanjing University of Information Science and Technology（南京信息工程大学）；Japan Advanced Institute of Science and Technology（日本科学技术大学）；The University of Sydney（悉尼大学）；University of Copenhagen（哥本哈根大学）；TU Dortmund University（多特蒙德技术大学）；Research Center Trustworthy Data Science and Security（可信数据科学与安全研究中心）
Artificial social influence via human-embodied AI agent interaction in immersive virtual reality (VR): Effects of similarity-matching during health conversations
Empathetic Conversational Agents: Utilizing Neural and Physiological Signals for Enhanced Empathetic Interactions
Empathic Computing Lab, Auckland Bioengineering Institute, University of Auckland（同理心计算实验室，奥克兰生物工程研究所，奥克兰大学）；Department of Psychological Medicine, Faculty of Health Psychology, University of Auckland（心理医学系，健康心理学学院，奥克兰大学）
ChatMuse: Supporting In-Person Small-Group Conversation Experience with a Proactive Assistive AI Agent in Mixed Reality
Florida International University（佛罗里达国际大学）；University of Texas at Dallas（德克萨斯大学达拉斯分校）；University of Michigan（密歇根大学）；University of Arizona（亚利桑那大学）；University of Cincinnati（辛辛那提大学）
Virtual reality (VR) is increasingly used as a platform for social interaction, including as a means for older adults to maintain engagement. However, there has been limited research to examine the features of social VR that are most relevant to older adults experiences. The current study was conducted to qualitatively analyze the behavior of older adults in a collaborative VR environment and evaluate aspects of design that affected their engagement outcomes. We paired 36 participants over the age of 60, from three diverse geographic locations, and asked them to interact in collaborative VR mo…
