arXiv 2508.096002025-09-03cs.SD

OSUM-EChat：通过理解驱动的口语对话增强端到端共情口语聊天机器人

OSUM-EChat: Enhancing End-to-End Empathetic Spoken Chatbot via Understanding-Driven Spoken Dialogue

Xuelong Geng, Qijie Shao, Hongfei Xue, Shuiyuan Wang, Hanke Xie, Zhao Guo, Yi Zhao, Guojian Li, Wenjie Tian, Chengyou Wang, Zhixian Zhao, Kangxiang Xia, Ziyu Zh… 展开作者

Xuelong Geng, Qijie Shao, Hongfei Xue, Shuiyuan Wang, Hanke Xie, Zhao Guo, Yi Zhao, Guojian Li, Wenjie Tian, Chengyou Wang, Zhixian Zhao, Kangxiang Xia, Ziyu Zhang, Zhennan Lin, Tianlun Zuo, Mingchen Shao, Yuang Cao, Guobin Ma, Longhao Li, Yuhang Dai, Dehui Gao, Dake Guo, Lei Xie

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对现有端到端口语对话系统共情能力不足、依赖大规模数据等问题，提出带三阶段训练策略和双思考机制的OSUM-EChat系统，配套发布EChat-200K数据集与EChat-eval基准，实验证明其共情响应表现更优。

摘要：

共情对于实现口语对话系统中的自然交互至关重要，它能让机器识别并恰当回应年龄、性别、情感等副语言线索。近期统一了语音理解与生成的端到端语音语言模型取得进展，提供了颇具前景的解决方案。但目前仍存在若干挑战：过度依赖大规模对话数据集、对传递共情至关重要的副语言线索提取不足，且缺乏面向共情的专用数据集与评估框架。为解决这些问题，我们提出OSUM-EChat——一款开源的端到端口语对话系统，旨在增强共情交互能力，尤其适用于资源受限场景。OSUM-EChat包含两项核心创新：（1）三阶段理解驱动的口语对话训练策略，将大型语音理解模型的能力拓展至口语对话任务；（2）语言-副语言双思考机制，通过思维链将副语言理解与对话生成相结合，使系统能生成更具共情性的回复。该方法在降低对大规模对话数据集依赖的同时，仍能保持高质量的共情交互。此外，我们还构建了EChat-200K数据集（一个丰富的共情语音到语音对话语料库）以及EChat-eval基准（一个用于评估对话系统共情能力的综合框架）。实验结果表明，OSUM-EChat在共情响应能力方面优于现有端到端口语对话模型，验证了其有效性。

英文摘要：

Empathy is crucial in enabling natural interactions within spoken dialogue systems, allowing machines to recognize and respond appropriately to paralinguistic cues such as age, gender, and emotion. Recent advancements in end-to-end speech language models, which unify speech understanding and generation, provide promising solutions. However, several challenges persist, including an over-reliance on large-scale dialogue datasets, insufficient extraction of paralinguistic cues vital for conveying empathy, and the lack of empathy-specific datasets and evaluation frameworks. To address these issues, we introduce OSUM-EChat, an open-source, end-to-end spoken dialogue system designed to enhance empathetic interactions, particularly in resource-limited settings. OSUM-EChat introduces two key innovations: (1) a three-stage understanding-driven spoken dialogue training strategy that extends the capabilities of a large speech understanding model to spoken dialogue tasks, and (2) a linguistic-paralinguistic dual thinking mechanism that integrates paralinguistic understanding through a chain of thought with dialogue generation, enabling the system to produce more empathetic responses. This approach reduces reliance on large-scale dialogue datasets while maintaining high-quality empathetic interactions. Additionally, we introduce the EChat-200K dataset, a rich corpus of empathetic speech-to-speech dialogues, and the EChat-eval benchmark, a comprehensive framework for evaluating the empathetic capabilities of dialogue systems. Experimental results demonstrate that OSUM-EChat outperforms end-to-end spoken dialogue models regarding empathetic responsiveness, validating its effectiveness.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。