Empathy Omni：通过大型语言模型实现共情语音响应生成
Empathy Omni: Enabling Empathetic Speech Response Generation through Large Language Models
AI总结：
本文提出Emotion Omni模型，通过构建20万条情感对话数据集，在数据有限且无需大规模预训练的条件下，使语音LLMs能够理解用户情感并生成共情语音响应，在语音质量和共情能力上超越现有模型。
摘要：
随着语音大型语言模型（speech LLMs）的发展，用户现在可以直接通过语音与助手交互。然而，大多数现有模型仅将响应内容转换为语音，而未能充分捕捉用户查询中丰富的情感线索，即同一句话可能因表达方式不同而传达不同含义。因此，情感理解对于改善人机交互至关重要。大多数共情语音LLMs依赖海量数据集，要求高昂的计算成本。一个关键挑战是，在数据有限且不进行大规模训练的情况下，构建能够生成共情响应的模型。为此，我们提出Emotion Omni，该模型能理解用户语音中的情感内容并生成共情响应。我们进一步开发了数据流水线，构建了20万条情感对话数据集，以支持共情语音助手。实验表明，Emotion Omni在不进行大规模预训练的情况下实现了可比的指令遵循能力，同时在语音质量（UTMOS:4.41）和共情能力（Emotion GPT Score: 3.97）上超越了现有模型。这些结果证实了其在语音保真度和情感表达方面的提升。演示可在https://w311411.github.io/omni_demo/获取。
英文摘要：
With the development of speech large language models (speech LLMs), users can now interact directly with assistants via speech. However, most existing models only convert response content into speech without fully capturing the rich emotional cues in user queries, where the same sentence may convey different meanings depending on the expression. Emotional understanding is thus essential for improving human-machine interaction. Most empathetic speech LLMs rely on massive datasets, demanding high computational cost. A key challenge is to build models that generate empathetic responses with limited data and without large-scale training. To this end, we propose Emotion Omni, a model that understands emotional content in user speech and generates empathetic responses. We further developed a data pipeline to construct a 200k emotional dialogue dataset supporting empathetic speech assistants. Experiments show that Emotion Omni achieves comparable instruction-following ability without large-scale pretraining, while surpassing existing models in speech quality (UTMOS:4.41) and empathy (Emotion GPT Score: 3.97). These results confirm its improvements in both speech fidelity and emotional expressiveness. Demos are available at https://w311411.github.io/omni_demo/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
