迈向单个语音语言模型中的高效语音-文本联合解码
Towards Efficient Speech-Text Jointly Decoding within One Speech Language Model
AI总结：
本文提出了一种早停交错解码方法，以提升语音-文本联合解码的效率和性能，并通过高质量QA数据集进一步优化语音问答性能。
摘要：
语音语言模型（语音LM）能够在单一模型中实现端到端的语音-文本建模，为语音对话系统提供有前途的方向。语音-文本联合解码范式的选择在性能、效率和对齐质量方面起着关键作用。在本工作中，我们系统地比较了代表性的联合语音-文本解码策略，包括交错和并行生成范式，在相同的基语言模型、语音分词器和训练数据的受控实验设置下。我们的结果表明，交错方法实现了最佳的对齐。然而，它由于长的token序列长度而导致推理缓慢。为了解决这个问题，我们提出了一种新的早停交错（ESI）模式，不仅显著加快了解码速度，还略微提高了性能。此外，我们还整理了高质量的问题回答（QA）数据集，以进一步提高语音QA性能。
英文摘要：
Speech language models (Speech LMs) enable end-to-end speech-text modeling within a single model, offering a promising direction for spoken dialogue systems. The choice of speech-text jointly decoding paradigm plays a critical role in performance, efficiency, and alignment quality. In this work, we systematically compare representative joint speech-text decoding strategies, including the interleaved, and parallel generation paradigms, under a controlled experimental setup using the same base language model, speech tokenizer and training data. Our results show that the interleaved approach achieves the best alignment. However it suffers from slow inference due to long token sequence length. To address this, we propose a novel early-stop interleaved (ESI) pattern that not only significantly accelerates decoding but also yields slightly better performance. Additionally, we curate high-quality question answering (QA) datasets to further improve speech QA performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
