HybridVC：基于文本和音频提示的高效语音风格转换
HybridVC: Efficient Voice Style Conversion with Text and Audio Prompts
AI总结：
本文提出基于预训练CVAE并结合对比学习的语音转换框架HybridVC，通过文本和音频提示实现灵活的多模态语音风格转换，在有限资源下具备卓越训练效率与广泛的应用潜力。
摘要：
我们提出了 HybridVC，这是一个基于预训练条件变分自编码器（CVAE）构建的语音转换（VC）框架，它结合了潜在模型与对比学习的优势。HybridVC 支持文本和音频提示，从而实现更灵活的语音风格转换。HybridVC 对以预训练说话人编码器获取的说话人嵌入为条件的潜在分布进行建模，并通过并行的对比学习优化风格文本嵌入，使其与说话人风格信息对齐。因此，HybridVC 能在有限的计算资源下被高效训练。我们的实验证明了 HybridVC 卓越的训练效率及其在高级多模态语音风格转换方面的能力。这突显了其在各种社交媒体平台中的用户自定义个性化语音等广泛应用的潜力。一项全面的消融研究进一步验证了我们方法的有效性。
英文摘要：
We introduce HybridVC, a voice conversion (VC) framework built upon a pre-trained conditional variational autoencoder (CVAE) that combines the strengths of a latent model with contrastive learning. HybridVC supports text and audio prompts, enabling more flexible voice style conversion. HybridVC models a latent distribution conditioned on speaker embeddings acquired by a pretrained speaker encoder and optimises style text embeddings to align with the speaker style information through contrastive learning in parallel. Therefore, HybridVC can be efficiently trained under limited computational resources. Our experiments demonstrate HybridVC's superior training efficiency and its capability for advanced multi-modal voice style conversion. This underscores its potential for widespread applications such as user-defined personalised voice in various social media platforms. A comprehensive ablation study further validates the effectiveness of our method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
