SoCodec：一种面向高效语言模型文本到语音合成的语义有序多流语音编解码器
SoCodec: A Semantic-Ordered Multi-Stream Speech Codec for Efficient Language Model Based Text-to-Speech Synthesis
AI总结：
本文提出语义有序多流语音编解码器 SoCodec，将语音压缩为更短的多流离散语义序列，并结合有序乘积量化与多流延迟语言模型，在帧移压缩 12 倍的情况下仍优于基线，实现高效 LM 基 TTS。
摘要：
长语音序列一直困扰着基于语言模型（LM）的文本到语音合成（TTS）方法，主要体现在建模复杂度和效率方面。本文提出 SoCodec，一种语义有序的多流语音编解码器，以解决该问题。它将语音压缩为更短的多流离散语义序列，每一帧包含多个 token。同时，提出有序乘积量化方法，将该序列约束为有序表示。该方法可与多流延迟语言模型结合，在 TTS 中沿时间轴和流轴实现更好的自回归生成。实验结果有力地证明了所提方法的有效性：即使将语音帧移从 20ms 压缩到 240ms（12 倍），仍能取得优于基线系统的性能。消融研究进一步验证了在学习所提出的有序多流语义表示以追求更短语音序列、实现高效基于 LM 的 TTS 中的重要性。
英文摘要：
The long speech sequence has been troubling language models (LM) based TTS approaches in terms of modeling complexity and efficiency. This work proposes SoCodec, a semantic-ordered multi-stream speech codec, to address this issue. It compresses speech into a shorter, multi-stream discrete semantic sequence with multiple tokens at each frame. Meanwhile, the ordered product quantization is proposed to constrain this sequence into an ordered representation. It can be applied with a multi-stream delayed LM to achieve better autoregressive generation along both time and stream axes in TTS. The experimental result strongly demonstrates the effectiveness of the proposed approach, achieving superior performance over baseline systems even if compressing the frameshift of speech from 20ms to 240ms (12x). The ablation studies further validate the importance of learning the proposed ordered multi-stream semantic representation in pursuing shorter speech sequences for efficient LM-based TTS.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
