ProsoCodec：面向韵律的语音编解码器用于语音转换
ProsoCodec: Prosody-Oriented Speech Codec for Voice Conversion
AI总结：
提出ProsoCodec，通过将韵律建模为条件残差而非分离流，在语音转换中改善韵律保留并减少源音色泄漏。
摘要：
神经语音编解码器高效压缩语音，并已成为语音生成的基础，但它们通常学习为整体表示，交织了语言内容、说话人身份和韵律。虽然这种设计对零样本语音克隆有效，但它阻碍了需要韵律保留或转移的下游任务，如语音转换。为解决此问题，我们引入了ProsoCodec，一种面向韵律的语音编解码器，将韵律建模为条件残差而非分离流。具体来说，通过将编码器和解码器都条件于文本和说话人嵌入作为前缀令牌，离散瓶颈被鼓励捕捉内容和说话人无法解释的韵律变化。为进一步保留韵律，我们使用低频梅尔频带并在配对同说话人话语上训练模型。语音转换实验表明，韵律保留得到改善，源音色泄漏减少。
英文摘要：
Neural speech codecs efficiently compress speech and have become a foundation for speech generation, but they are typically learned as holistic representations that intertwine linguistic content, speaker identity, and prosody. While this design is effective for zero-shot voice cloning, it hinders downstream tasks that require prosody preservation or transfer, such as voice conversion. To address this, we introduce ProsoCodec, a prosody-oriented speech codec that models prosody as a conditional residual rather than as a disentangled stream. Specifically, by conditioning both the encoder and decoder on text and speaker embeddings as prefix tokens, the discrete bottleneck is encouraged to capture prosodic variation not explained by content and speaker. To further preserve prosody, we use the low-frequency mel band and train the model on paired same-speaker utterances. Experiments on voice conversion show improved prosody preservation and reduced source-timbre leakage.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
