SAC：具有语义-声学双流量化神经语音编解码器
SAC: Neural Speech Codec with Semantic-Acoustic Dual-Stream Quantization
AI总结：
SAC通过语义-声学双流量化提升语音编解码器的重建与语义表示能力，实现更自然的语音生成和更高效的文本到语音转换。
摘要：
将连续语音信号转换为离散令牌的语音编解码器已成为语音语言模型的关键组件。然而，现有编解码器在平衡高质量重建与语义丰富表示方面存在困难，限制了其在生成和理解任务中的有效性。在本工作中，我们提出了SAC，一种具有语义-声学双流量化的神经语音编解码器。通过将语义和声学建模拆分为两个专用流，SAC使每个流都能针对其各自的角色进行优化。全面评估显示，SAC在不同比特率下在干净和嘈杂条件下均实现了强大的重建性能，特别是在UTMOS和WER上得分特别高，表明其具有更自然的自然度和可理解性。此外，SAC在语义表示上显著优于先前的编解码器，接近连续自监督嵌入的水平。当用作基于LLM的文本到语音的分词器时，SAC能够实现一个单阶段自回归（AR）TTS模型，其性能明显优于最先进的AR系统。我们的解耦分析进一步验证了双流设计的有效性，为可控语音生成提供了新的潜力。
英文摘要：
Speech codecs that convert continuous speech signals into discrete tokens have become essential for speech language models. However, existing codecs struggle to balance high-quality reconstruction with semantically rich representations, limiting their effectiveness in both generative and understanding tasks. In this work, we propose SAC, a neural speech codec with semantic-acoustic dual-stream quantization. By disentangling semantic and acoustic modeling into two dedicated streams, SAC enables each to be optimized for its respective role. Comprehensive evaluations show that SAC achieves strong reconstruction performance across diverse bitrates under both clean and noisy conditions, with particularly high scores on UTMOS and WER, indicating superior naturalness and intelligibility. Moreover, SAC substantially surpasses prior codecs in semantic representation, approaching the level of continuous self-supervised embeddings. When used as a tokenizer for LLM-based text-to-speech, SAC enables a single-stage autoregressive (AR) TTS model that clearly outperforms state-of-the-art AR systems. Our disentanglement analysis further validates the effectiveness of the dual-stream design, offering new potential for controllable speech generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
