A$^2$-LLM：一种端到端的对话音频虚拟形象大语言模型
A$^2$-LLM: An End-to-end Conversational Audio Avatar Large Language Model
AI总结：
A$^2$-LLM通过统一框架联合推理语言、音频语调和3D面部运动，实现端到端的情感表达对话虚拟形象，提升实时效率与情感表现力。
摘要：
开发具有表现力和响应性的对话数字人类是下一代人机交互的关键。尽管大型语言模型（LLMs）显著增强了对话能力，但当前大多数系统仍依赖于连接独立模块的级联架构。这些流水线常常受到累积误差、高延迟和较差的实时性能的困扰。由于缺乏对底层对话上下文的访问，这些流水线本质上优先考虑僵化的嘴唇同步而非情感深度。为了解决这些挑战，我们提出了A$^2$-LLM，一种端到端的对话音频虚拟形象大语言模型，该模型在一个统一的框架内联合推理语言、音频语调和3D面部运动。为了促进训练，我们引入了FLAME-QA，一个高质量的多模态数据集，旨在以问答格式对齐语义意图与表达性面部动态。通过利用深度语义理解，A$^2$-LLM生成了超越简单嘴唇同步的富有情感的面部运动。实验结果表明，我们的系统在保持实时效率（500毫秒延迟，0.7 RTF）的同时，实现了更优越的情感表现力。
英文摘要：
Developing expressive and responsive conversational digital humans is a cornerstone of next-generation human-computer interaction. While large language models (LLMs) have significantly enhanced dialogue capabilities, most current systems still rely on cascaded architectures that connect independent modules. These pipelines are often plagued by accumulated errors, high latency, and poor real-time performance. Lacking access to the underlying conversational context, these pipelines inherently prioritize rigid lip-sync over emotional depth. To address these challenges, we propose A$^2$-LLM, an end-to-end conversational audio avatar large language model that jointly reasons about language, audio prosody, and 3D facial motion within a unified framework. To facilitate training, we introduce FLAME-QA, a high-quality multimodal dataset designed to align semantic intent with expressive facial dynamics within a QA format. By leveraging deep semantic understanding, A$^2$-LLM generates emotionally rich facial movements beyond simple lip-synchronization. Experimental results demonstrate that our system achieves superior emotional expressiveness while maintaining real-time efficiency (500 ms latency, 0.7 RTF).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
