Large Language Models (LLMs) have demonstrated remarkable capabilities in conversational tasks. Embodying an LLM as a virtual human allows users to engage in face-to-face social interactions in Virtual Reality. However, the influence of person- and task-related factors in social interactions with LLM-controlled agents remains unclear. In this study, forty-six participants interacted with a virtual agent whose persona was manipulated as extravert or introvert in three different conversational tasks (small talk, knowledge test, convincing). Social-evaluation, emotional experience, and realism were assessed using ratings. Interactive engagement was measured by quantifying participants' words and conversational turns. Finally, we measured participants' willingness to ask the agent for help during the knowledge test. Our findings show that the extraverted agent was more positively evaluated, elicited a more pleasant experience and greater engagement, and was assessed as more realistic compared to the introverted agent. Whereas persona did not affect the tendency to ask for help, participants were generally more confident in the answer when they had help of the LLM. Variation of personality traits of LLM-controlled embodied virtual agents, therefore, affects social-emotional processing and behavior in virtual interactions. Embodied virtual agents allow the presentation of naturalistic social encounters in a virtual environment.
IBM Research AI（IBM研究院人工智能部门）；IBM Argentina（IBM阿根廷公司）
The Programmer's Assistant is an experimental prototype software development environment that integrates a chatbot with a code editor. Conversational capability was achieved by using an existing code-fluent Large Language Model and providing it with a prompt that establishes a conversational interaction pattern, a set of conventions, and a style of interaction appropriate for the application. A discussion of the evolution of the prompt provides a case study in how to coax an existing foundation model to behave in a desirable manner for a particular application.
A Personalized Conversational Benchmark: Towards Simulating Personalized Conversations
University of Southern California（南加州大学）；Adobe Research（Adobe研究院）；University of California, San Diego（加州大学圣地亚哥分校）；Intel AI Research（英特尔人工智能研究）；University of Maryland, College Park（马里兰大学学院市分校）；Vanderbilt University（范德比大学）；Virginia Polytechnic Institute and State University（弗吉尼亚理工学院和州立大学）；University of Notre Dame（诺特丹大学）；University of Oregon（俄勒冈大学）；University of Illinois Chicago（伊利诺伊大学芝加哥分校）；Arizona State University（亚利桑那州立大学）
Towards the "Digital Me": A vision of authentic Conversational Agents powered by personal Human Digital Twins
Technical University of Denmark（丹麦技术大学）；School of Design（设计学院）；University of Strathclyde（斯特拉斯克莱德大学）；Department of Design, Manufacturing and Engineering Management（设计、制造与工程管理系）
One Size doesn't Fit All: A Personalized Conversational Tutoring Agent for Mathematics Instruction
School of Computer Science, Wuhan University（武汉大学计算机学院）；DAMO Academy, Alibaba Group（阿里巴巴达摩院）；College of Computer and Cyber Security, Hebei Normal University（河北师范大学计算机与网络空间安全学院）
针对数学教学中学习者差异被忽视的问题，本文提出 PACE 个性化对话辅导智能体，结合学习风格画像、苏格拉底式反馈与个性化训练，提升教学适应性和学习效果。
ConvoDrift: A Multi-Turn Conversational Dataset for Modeling Stylistic Tone Evolution
Informatics Institute of Technology, Sri Lanka（斯里兰卡信息技术学院）；National Health and Medical Research Council, Australia（澳大利亚国家健康与医学研究理事会）；Luxembourg Centre for Systems Biomedicine (LCSB), University of Luxembourg（卢森堡大学系统生物医学中心）
There is a growing recognition that artists use valuable ways to understand and work with cognitive and perceptual mechanisms to convey desired experiences and narrative in their created artworks (DiPaola et al., 2010; Zeki, 2001). This paper documents our attempt to computationally model the creative process of a portrait painter, who relies on understanding human traits (i.e., personality and emotions) to inform their art. Our system includes an empathic conversational interaction component to capture the dominant personality category of the user and a generative AI Portraiture system that u…
The creation and support of Embodied Conversational Agents (ECAs) has been quite challenging, as features required might not be straight-forward to implement and to integrate in a single application. Furthermore, ECAs as desktop applications present drawbacks for both developers and users; the former have to develop for each device and operating system and the latter must install additional software, limiting their widespread use. In this paper we demonstrate how recent advances in web technologies show promising steps towards capable web-based ECAs, through some off-the-shelf technologies, in…
Visual appearance is an important aspect influencing the perception and consequent acceptance of Embodied Conversational Agents (ECA). To this end, the Uncanny Valley theory contradicts the common assumption that increased humanization of characters leads to better acceptance. Rather, it shows that anthropomorphic behavior may trigger feelings of eeriness and rejection in people. The work presented in this paper explores whether four different autonomous ECAs, specifically build for a European research project, are affected by this effect, and how they compare to two slightly more realisticall…
The use of digital resources has been increasing in every instance of todays society, being it in business or even ludic purposes. Despite such ever increasing use of technologies as interfaces, in all fields, it seems that it lacks the importance of users perception in this context. This work aims to present a case study about the evaluation of ECAs. We propose a Long-Term Interaction (LTI) to evaluate our conversational agent effectiveness through the user perception and compare it with Short-Term Interactions (STIs), performed by three users. Results show that many different aspects of user…
Non-verbal behavior is essential for embodied agents like social robots, virtual avatars, and digital humans. Existing behavior authoring approaches including keyframe animation and motion capture are too expensive to use when there are numerous utterances requiring gestures. Automatic generation methods show promising results, but their output quality is not satisfactory yet, and it is hard to modify outputs as a gesture designer wants. We introduce a new gesture generation toolkit, named SGToolkit, which gives a higher quality output than automatic methods and is more efficient than manual a…
Embodied Conversational Agents that make use of co-speech gestures can enhance human-machine interactions in many ways. In recent years, data-driven gesture generation approaches for ECAs have attracted considerable research attention, and related methods have continuously improved. Real-time interaction is typically used when researchers evaluate ECA systems that generate rule-based gestures. However, when evaluating the performance of ECAs based on data-driven methods, participants are often required only to watch pre-recorded videos, which cannot provide adequate information about what a pe…
Storytelling is fundamental to language, including culture, conversation and communication in their broadest senses. It thus emerges as an essential component of intelligent systems, including systems where natural language is not a primary focus or where we do not usually think of a story being involved. In this paper we explore the emergence of storytelling as a requirement in embodied conversational agents, including its role in educational and health interventions, as well as in a general-purpose computer interface for people with disabilities or other constraints that prevent the use of t…
Embodied conversational agents (ECA) are often designed to produce nonverbal behavior to complement or enhance their verbal communication. One such form of nonverbal behavior is co-speech gesturing, which involves movements that the agent makes with its arms and hands that are paired with verbal communication. Co-speech gestures for ECAs can be created using different generation methods, divided into rule-based and data-driven processes, with the latter gaining traction because of the increasing interest from the applied machine learning community. However, reports on gesture generation method…
We present the first systematic analysis of personality dimensions developed specifically to describe the personality of speech-based conversational agents. Following the psycholexical approach from psychology, we first report on a new multi-method approach to collect potentially descriptive adjectives from 1) a free description task in an online survey (228 unique descriptors), 2) an interaction task in the lab (176 unique descriptors), and 3) a text analysis of 30,000 online reviews of conversational agents (Alexa, Google Assistant, Cortana) (383 unique descriptors). We aggregate the results…
