E-chat: 基于大型语言模型的情感敏感型语音对话系统
E-chat: Emotion-sensitive Spoken Dialogue System with Large Language Models
AI总结：
针对语音对话系统无法基于情感语音生成合适回应的问题，提出情感敏感型语音对话系统E-chat，利用语音编码器提取情感嵌入并结合大型语言模型，同时构建E-chat200数据集，在多项指标上优于基线模型。
摘要：
本研究关注人机语音交互中的情感敏感语音对话。随着大型语言模型（LLMs）的进步，对话系统能够处理包括音频在内的多模态数据。近期模型通过整合各种音频事件，增强了对复杂音频信号的理解。然而，它们无法基于带有情感的语音生成合适的回应。为解决此问题，我们提出了Emotional chat Model（E-chat），这是一种新型语音对话系统，能够理解并回应语音传达的情感。该模型利用语音编码器提取的情感嵌入，与LLMs结合，使其能根据不同情感语境做出回应。此外，我们推出了专为情感敏感语音对话设计的E-chat200数据集。在多项评估指标中，E-chat始终优于基线模型，展现了其在情感理解和人机交互方面的潜力。
英文摘要：
This study focuses on emotion-sensitive spoken dialogue in human-machine speech interaction. With the advancement of Large Language Models (LLMs), dialogue systems can handle multimodal data, including audio. Recent models have enhanced the understanding of complex audio signals through the integration of various audio events. However, they are unable to generate appropriate responses based on emotional speech. To address this, we introduce the Emotional chat Model (E-chat), a novel spoken dialogue system capable of comprehending and responding to emotions conveyed from speech. This model leverages an emotion embedding extracted by a speech encoder, combined with LLMs, enabling it to respond according to different emotional contexts. Additionally, we introduce the E-chat200 dataset, designed explicitly for emotion-sensitive spoken dialogue. In various evaluation metrics, E-chat consistently outperforms baseline model, demonstrating its potential in emotional comprehension and human-machine interaction.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
