FreezeEmpath: 基于冻结大语言模型的高效共情语音聊天机器人训练
FreezeEmpath: Efficient Training for Empathetic Spoken Chatbots with Frozen LLMs
AI总结：
本文提出 FreezeEmpath，通过冻结大语言模型参数，利用现有语音指令和情感识别数据高效训练共情语音聊天机器人，实现情感表达和对话性能的提升。
摘要：
共情对于促进语音对话系统中的自然交互至关重要，因为它使机器能够识别人类语音的情感基调并提供共情响应。近期研究在基于大语言模型（LLMs）的共情语音聊天机器人方面取得了显著进展。然而，在训练此类模型时仍存在挑战，包括依赖昂贵的共情语音指令数据和生成语音的情感表达不足。对LLM进行跨模态共情指令数据微调可能导致灾难性遗忘和一般能力下降。为了解决这些挑战，我们提出了 FreezeEmpath，一种通过简单高效的方式训练的端到端共情语音聊天机器人。整个训练过程仅依赖于现有语音指令数据和语音情感识别（SER）数据，同时保持LLM参数冻结。实验表明，FreezeEmpath能够生成情感丰富的语音，并在共情对话、SER和SpokenQA任务中优于其他共情模型，证明了我们的训练策略的有效性。
英文摘要：
Empathy is essential for fostering natural interactions in spoken dialogue systems, as it enables machines to recognize the emotional tone of human speech and deliver empathetic responses. Recent research has made significant progress in developing empathetic spoken chatbots based on large language models (LLMs). However, several challenges still exist when training such models, including reliance on costly empathetic speech instruction data and a lack of emotional expressiveness in the generated speech. Finetuning LLM with cross-modal empathetic instruction data may also lead to catastrophic forgetting and a degradation of its general capability. To address these challenges, we propose FreezeEmpath, an end-to-end empathetic spoken chatbot trained in a simple and efficient manner. The entire training process relies solely on existing speech instruction data and speech emotion recognition (SER) data, while keeping the LLM's parameters frozen. Experiments demonstrate that FreezeEmpath is able to generate emotionally expressive speech and outperforms other empathetic models in empathetic dialogue, SER, and SpokenQA tasks, demonstrating the effectiveness of our training strategy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
