Maia:一种用于人机交互的实时非语言聊天
Maia: A Real-time Non-Verbal Chat for Human-AI Interaction
AI总结:
本文提出实时非语言人机交互系统 Maia,结合检索、统计与深度学习及艺术化情感表达,通过二维扩散模型与三维化身实验验证其可提升交互自然性和参与感。
摘要:
在计算机视觉中对面对面交流进行建模,重点是识别和分析交互过程中的非语言线索与行为,这为我们提出的替代基于文本的人机交互方案奠定了基础。通过借助面部表情、头部和身体动作等非语言视觉交流,我们旨在通过一种超越手势镜像的新颖即兴元素来增强参与感并吸引用户注意力。我们的目标是实时跟踪和分析面部表情及其他非语言线索,并利用这些信息构建能够预测和理解人类行为的模型。我们的方法可实时运行且只需极少的计算资源,标志着在使人机交互更自然、更易获取方面的重大飞跃。我们提供三种互补方法,分别基于检索、统计和深度学习技术。本工作的一项关键创新是在高效的人机交互系统之上整合艺术成分,将艺术作为传递情感的媒介。我们的方法并不局限于特定艺术形式,可适配各种绘画、动画和虚拟化身。在实验中,我们将作为二维情感转译媒介的最先进扩散模型,与本文提出的三维虚拟化身 Maia 进行比较;Maia 不仅具有面部动作,还具有身体运动,以带来更自然、更具吸引力的体验。我们通过人工评估和自动评估流程证明了本方法在将 AI 生成的情感转译为人类可共情的表情方面的有效性,突显了其在各类应用中显著提升人机交互自然性和参与度的潜力。
英文摘要:
Modeling face-to-face communication in computer vision, which focuses on recognizing and analyzing nonverbal cues and behaviors during interactions, serves as the foundation for our proposed alternative to text-based Human-AI interaction. By leveraging nonverbal visual communication, through facial expressions, head and body movements, we aim to enhance engagement and capture the user's attention through a novel improvisational element, that goes beyond mirroring gestures. Our goal is to track and analyze facial expressions, and other nonverbal cues in real-time, and use this information to build models that can predict and understand human behavior. Operating in real-time and requiring minimal computational resources, our approach signifies a major leap forward in making AI interactions more natural and accessible. We offer three different complementary approaches, based on retrieval, statistical, and deep learning techniques. A key novelty of our work is the integration of an artistic component atop an efficient human-computer interaction system, using art as a medium to transmit emotions. Our approach is not art-specific and can be adapted to various paintings, animations, and avatars. In our experiments, we compare state-of-the-art diffusion models as mediums for emotion translation in 2D, and our 3D avatar, Maia, that we introduce in this work, with not just facial movements but also body motions for a more natural and engaging experience. We demonstrate the effectiveness of our approach in translating AI-generated emotions into human-relatable expressions, through both human and automatic evaluation procedures, highlighting its potential to significantly enhance the naturalness and engagement of Human-AI interactions across various applications.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。