多模态情绪条件作用与情感一致性对具身对话智能体的重要性
The Importance of Multimodal Emotion Conditioning and Affect Consistency for Embodied Conversational Agents
AI总结：
本文提出 ACTOR 情感一致多模态行为生成框架，并通过199人用户研究证明情绪条件作用与跨模态情感一致可显著提升具身对话智能体的情感感知与表现力。
摘要：
以往关于具身虚拟智能体情绪感知的研究表明，使用虚拟角色通过与人类互动来传达情绪是有效的。然而，创建一个具有表现性行为的自主具身对话智能体面临两大挑战。第一个挑战是难以综合出每种模态下与真实人类行为一样富有表现力的对话行为。第二个挑战是情感被独立建模，这使得难以生成在所有模态中具有一致情绪的多模态响应。在本研究中，我们提出了一个概念框架 ACTOR（Affect-Consistent mulTimodal behaviOR generation，情感一致的多模态行为生成），旨在通过生成以一致的驱动情感为条件的多模态行为来增强情感感知。我们开展了一项包含199名参与者的用户研究，以评估普通人如何判断从相对于驱动情感一致和不一致的多模态行为中感知到的情感。结果表明，在所有模型条件中，我们的情感一致框架在驱动情感感知方面获得了最高的 Likert 评分。统计分析表明，使某一模态的情感不一致会显著降低对驱动情感的感知。我们还观察到，与情感不一致的行为相比，以一致情感为条件的多模态行为更具表现力。因此，我们得出结论，多模态情绪条件作用和情感一致性对于增强具身对话智能体的情感感知至关重要。
英文摘要：
Previous studies regarding the perception of emotions for embodied virtual agents have shown the effectiveness of using virtual characters in conveying emotions through interactions with humans. However, creating an autonomous embodied conversational agent with expressive behaviors presents two major challenges. The first challenge is the difficulty of synthesizing the conversational behaviors for each modality that are as expressive as real human behaviors. The second challenge is that the affects are modeled independently, which makes it difficult to generate multimodal responses with consistent emotions across all modalities. In this work, we propose a conceptual framework, ACTOR (Affect-Consistent mulTimodal behaviOR generation), that aims to increase the perception of affects by generating multimodal behaviors conditioned on a consistent driving affect. We have conducted a user study with 199 participants to assess how the average person judges the affects perceived from multimodal behaviors that are consistent and inconsistent with respect to a driving affect. The result shows that among all model conditions, our affect-consistent framework receives the highest Likert scores for the perception of driving affects. Our statistical analysis suggests that making a modality affect-inconsistent significantly decreases the perception of driving affects. We also observe that multimodal behaviors conditioned on consistent affects are more expressive compared to behaviors with inconsistent affects. Therefore, we conclude that multimodal emotion conditioning and affect consistency are vital to enhancing the perception of affects for embodied conversational agents.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
