PURE 编码器:残差熵的渐进展开用于语音编码器学习
PURE Codec: Progressive Unfolding of Residual Entropy for Speech Codec Learning
AI总结:
PURE编码器通过预训练语音增强模型指导多阶段量化,提升低比特率语音编码的训练稳定性与重建质量,尤其在嘈杂环境下表现优异。
摘要:
神经语音编码器在低比特率压缩中实现了强劲的性能,但残差向量量化(RVQ)常面临训练不稳定和分解无效的问题,限制了重建质量和效率。我们提出了PURE编码器(残差熵的渐进展开),这是一种新的框架,通过预训练的语音增强模型指导多阶段量化。第一阶段量化重构低熵、去噪的语音嵌入,后续阶段编码残差高熵组件。这种设计显著提高了训练稳定性。实验表明,PURE在重建和下游语音语言模型基于的文本到语音任务中一致优于传统基于RVQ的编码器,特别是在嘈杂训练条件下。
英文摘要:
Neural speech codecs have achieved strong performance in low-bitrate compression, but residual vector quantization (RVQ) often suffers from unstable training and ineffective decomposition, limiting reconstruction quality and efficiency. We propose PURE Codec (Progressive Unfolding of Residual Entropy), a novel framework that guides multi-stage quantization using a pre-trained speech enhancement model. The first quantization stage reconstructs low-entropy, denoised speech embeddings, while subsequent stages encode residual high-entropy components. This design improves training stability significantly. Experiments demonstrate that PURE consistently outperforms conventional RVQ-based codecs in reconstruction and downstream speech language model-based text-to-speech, particularly under noisy training conditions.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。