arXiv 2512.132512026-01-04cs.SD

DisCo-Speech: 基于解耦语音编解码器的可控零样本语音生成

DisCo-Speech: Controllable Zero-Shot Speech Generation with A Disentangled Speech Codec

China Mobile Nineverse Artificial Intelligence Technology (Beijing) Co., Ltd.（中国移动Nineverse人工智能技术（北京）有限公司）；Nineverse Institute of Artificial Intelligence（Nineverse人工智能研究院）；The State Key Laboratory of Multimedia Information Processing, Peking University, Beijing, China（多媒体信息处理国家重点实验室，北京大学，北京，中国）

Tao Li, Wenshuo Ge, Zhichao Wang, Zihao Cui, Yong Ma, Yingying Gao, Chao Deng, Shilei Zhang, Junlan Feng

备注：: Updated with 6,000 hours of additional training data and improved performance. Expanded appendix with ablation studies, training objectives, and hyperparameter settings for better reproducibility. Audio and code links included

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

DisCo-Speech通过解耦语音编解码器实现可控零样本语音生成，提升语音克隆与语调控制性能。

摘要：

基于编解码器的语言模型（LMs）已革新了文本到语音（TTS）领域。然而，标准编解码器将音色和语调纠缠在一起，这阻碍了基于延续的LMs中独立控制的能力。为解决这一挑战，我们提出了DisCo-Speech，一个具有解耦语音编解码器（DisCodec）和基于LM的生成器的零样本可控TTS框架。核心组件DisCodec采用两阶段设计：1）三因素解耦通过并行编码器和混合损失将语音分为内容、语调和音色子空间；2）融合与重建将内容和语调合并为统一的内容-语调标记，适用于LM预测，同时联合优化重建以解决解耦-重建权衡问题。这使LM能够从风格提示中进行语调延续，同时解码器注入目标音色，实现灵活的零样本控制。实验表明，DisCo-Speech在语音克隆和零样本语调控制方面表现优异。通过在编解码器层面解决核心纠缠问题，DisCo-Speech为可控语音合成提供了稳健的基础。

英文摘要：

Codec-based language models (LMs) have revolutionized text-to-speech (TTS). However, standard codecs entangle timbre and prosody, which hinders independent control in continuation-based LMs. To tackle this challenge, we propose DisCo-Speech, a zero-shot controllable TTS framework featuring a disentangled speech codec (DisCodec) and an LM-based generator. The core component DisCodec employs a two-stage design: 1) tri-factor disentanglement to separate speech into content, prosody, and timbre subspaces via parallel encoders and hybrid losses; and 2) fusion and reconstruction that merges content and prosody into unified content-prosody tokens suitable for LM prediction, while jointly optimizing reconstruction to address the disentanglement-reconstruction trade-off. This allows the LM to perform prosodic continuation from a style prompt while the decoder injects target timbre, enabling flexible zero-shot control. Experiments demonstrate that DisCo-Speech achieves competitive voice cloning and superior zero-shot prosody control. By resolving the core entanglement at the codec level, DisCo-Speech provides a robust foundation for controllable speech synthesis.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。