面向开放式端到端口语对话系统的思维链训练
Chain-of-Thought Training for Open E2E Spoken Dialogue Systems
AI总结：
本文提出利用思维链形式化训练开放式端到端口语对话系统，使对话训练与多模态语言模型的ASR、TTS和文本LM预训练保持一致，在仅300小时公开对话数据上实现超过1.5的ROUGE-1提升。
摘要：
与传统的级联流水线不同，端到端（E2E）口语对话系统保持完全可微性并捕捉非音素信息，使其非常适合建模口语交互。然而，现有的端到端方法通常需要大规模训练数据，并且生成的响应缺乏语义连贯性。我们提出一种简单而有效的策略，利用思维链（CoT）形式化，确保对话数据的训练与多模态语言模型（LM）在语音识别（ASR）、文本到语音合成（TTS）和文本语言模型任务上的预训练保持紧密一致。我们的方法相比基线实现了超过1.5的ROUGE-1提升，成功在公开的人类-人类对话数据集上训练口语对话系统，同时计算效率足够高，仅需300小时的公开人类-人类对话数据（如Switchboard）即可训练。我们将公开发布模型和训练代码。
英文摘要：
Unlike traditional cascaded pipelines, end-to-end (E2E) spoken dialogue systems preserve full differentiability and capture non-phonemic information, making them well-suited for modeling spoken interactions. However, existing E2E approaches often require large-scale training data and generates responses lacking semantic coherence. We propose a simple yet effective strategy leveraging a chain-of-thought (CoT) formulation, ensuring that training on conversational data remains closely aligned with the multimodal language model (LM)'s pre-training on speech recognition~(ASR), text-to-speech synthesis (TTS), and text LM tasks. Our method achieves over 1.5 ROUGE-1 improvement over the baseline, successfully training spoken dialogue systems on publicly available human-human conversation datasets, while being compute-efficient enough to train on just 300 hours of public human-human conversation data, such as the Switchboard. We will publicly release our models and training code.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
