AIVA：一款用于情感感知交互的基于AI的虚拟伴侣
AIVA: An AI-based Virtual Companion for Emotion-aware Interaction
AI总结：
针对大语言模型仅处理文本、缺乏非语言情感感知的问题，研究提出虚拟伴侣AIVA，通过多模态情感感知网络、情感提示工程及语音动画模块实现共情交互，为情感感知智能体提供应用框架。
摘要：
大语言模型（LLMs）的最新进展显著提升了自然语言理解与生成能力，优化了人机交互（HCI）。但LLMs仅局限于单模态文本处理，缺乏从非语言信号中解读情感线索的能力，阻碍了更具沉浸感与共情性的交互。本研究探索将多模态情感感知融入LLMs以打造情感感知智能体。我们提出AIVA——一款基于AI的虚拟伴侣，它能捕捉多模态情感线索，实现情感对齐且生动的人机交互。AIVA引入了多模态情感感知网络（MSPN），该网络采用跨模态融合Transformer与监督对比学习来提供情感线索。此外，我们开发了一种情感感知提示工程策略用于生成共情回应，并集成了文本转语音（TTS）系统与动画虚拟形象模块以实现富有表现力的交互。AIVA为情感感知智能体提供了一个框架，可应用于伴侣机器人、社会关怀、心理健康以及以人为本的AI领域。
英文摘要：
Recent advances in Large Language Models (LLMs) have significantly improved natural language understanding and generation, enhancing Human-Computer Interaction (HCI). However, LLMs are limited to unimodal text processing and lack the ability to interpret emotional cues from non-verbal signals, hindering more immersive and empathetic interactions. This work explores integrating multimodal sentiment perception into LLMs to create emotion-aware agents. We propose \ours, an AI-based virtual companion that captures multimodal sentiment cues, enabling emotionally aligned and animated HCI. \ours introduces a Multimodal Sentiment Perception Network (MSPN) using a cross-modal fusion transformer and supervised contrastive learning to provide emotional cues. Additionally, we develop an emotion-aware prompt engineering strategy for generating empathetic responses and integrate a Text-to-Speech (TTS) system and animated avatar module for expressive interactions. \ours provides a framework for emotion-aware agents with applications in companion robotics, social care, mental health, and human-centered AI.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
