利用思维链在没有对应问答数据的情况下实现共情口语对话
Leveraging Chain of Thought towards Empathetic Spoken Dialogue without Corresponding Question-Answering Data
AI总结：
针对口语对话缺乏问答数据导致共情响应不足的问题，提出LPE两阶段训练方法，结合思维链提示，首次实现基于语音的共情对话生成。
摘要：
共情对话对于自然的人机交互至关重要，它使对话系统能够以更加个性化和具有情感意识的方式做出回应，从而提高用户满意度和参与度。大型语言模型（LLM）的出现通过利用其强大的能力彻底改变了对话生成，并在多模态领域展示了其潜力。许多研究将语音与基于文本的LLM集成，以语音问题作为输入并输出文本响应。然而，缺乏包含语音风格信息的口语问答数据集用于监督微调（SFT）限制了这些系统的性能。因此，虽然这些系统擅长理解语音内容，但它们往往难以生成共情响应。为此，我们提出了一种新颖的方法，规避了对问答数据的需求，称为“聆听、感知与表达”（LPE）。我们的方法采用两阶段训练过程，首先引导LLM聆听内容并感知语音的情感方面。随后，我们利用思维链（CoT）提示来激发模型基于所聆听的口语内容和感知到的情感线索表达共情响应的潜力。我们通过实验证明了所提出方法的有效性。据我们所知，这是首次尝试将CoT用于基于语音的对话。
英文摘要：
Empathetic dialogue is crucial for natural human-computer interaction, allowing the dialogue system to respond in a more personalized and emotionally aware manner, improving user satisfaction and engagement. The emergence of large language models (LLMs) has revolutionized dialogue generation by harnessing their powerful capabilities and shown its potential in multimodal domains. Many studies have integrated speech with text-based LLMs to take speech question as input and output text response. However, the lack of spoken question-answering datasets that include speech style information to supervised fine-tuning (SFT) limits the performance of these systems. As a result, while these systems excel at understanding speech content, they often struggle to generate empathetic responses. In response, we propose a novel approach that circumvents the need for question-answering data, called Listen, Perceive, and Express (LPE). Our method employs a two-stage training process, initially guiding the LLM to listen the content and perceive the emotional aspects of speech. Subsequently, we utilize Chain-of-Thought (CoT) prompting to unlock the model's potential for expressing empathetic responses based on listened spoken content and perceived emotional cues. We employ experiments to prove the effectiveness of proposed method. To our knowledge, this is the first attempt to leverage CoT for speech-based dialogue.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
