ZSVC：基于解耦潜在扩散模型与对抗训练的零样本风格语音转换
ZSVC: Zero-shot Style Voice Conversion with Disentangled Latent Diffusion Models and Adversarial Training
AI总结：
提出 ZSVC 零样本风格语音转换方法，结合语音编解码器、语音提示潜在扩散、信息瓶颈、UMAdaIN 与对抗训练，实现风格与音色解耦，并在 44,000 小时数据上验证其优越性能。
摘要：
风格语音转换旨在将源语音的说话风格转换为所需风格，同时保留原始说话人的身份。然而，以往的风格语音转换方法主要关注情感等定义明确的领域，限制了其实际应用。在本研究中，我们提出 ZSVC，这是一种新颖的零样本风格语音转换方法，利用语音编解码器和带有语音提示机制的潜在扩散模型，以促进说话风格转换的上下文学习。为了解耦说话风格与说话人音色，我们引入信息瓶颈来过滤源语音中的说话风格，并采用不确定性建模自适应实例归一化（Uncertainty Modeling Adaptive Instance Normalization，UMAdaIN）来扰动风格提示中的说话人音色。此外，我们提出一种新颖的对抗训练策略，以增强上下文学习并提高风格相似度。在 44,000 小时语音数据上进行的实验表明，ZSVC 在零样本场景下生成具有多样说话风格的语音方面性能优越。
英文摘要：
Style voice conversion aims to transform the speaking style of source speech into a desired style while keeping the original speaker's identity. However, previous style voice conversion approaches primarily focus on well-defined domains such as emotional aspects, limiting their practical applications. In this study, we present ZSVC, a novel Zero-shot Style Voice Conversion approach that utilizes a speech codec and a latent diffusion model with speech prompting mechanism to facilitate in-context learning for speaking style conversion. To disentangle speaking style and speaker timbre, we introduce information bottleneck to filter speaking style in the source speech and employ Uncertainty Modeling Adaptive Instance Normalization (UMAdaIN) to perturb the speaker timbre in the style prompt. Moreover, we propose a novel adversarial training strategy to enhance in-context learning and improve style similarity. Experiments conducted on 44,000 hours of speech data demonstrate the superior performance of ZSVC in generating speech with diverse speaking styles in zero-shot scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
