SLMGAN：在GAN中利用语音语言模型表征实现无监督零样本语音转换
SLMGAN: Exploiting Speech Language Model Representations for Unsupervised Zero-Shot Voice Conversion in GANs
AI总结：
该研究提出SLMGAN方法，以StarGANv2-VC为基础加入基于SLM的WavLM判别器和SLM特征匹配损失，实现无监督零样本语音转换，自然度优于现有SOTA模型且相似度相当。
摘要：
近年来，大规模预训练语音语言模型（SLMs）在文本到语音合成、语音转换、语音增强等各类生成式语音建模应用中取得了显著进展。这类应用通常需将文本或语音输入映射到预训练SLM的表征，再从中解码出目标语音。本文提出一种新方法SLMGAN，在生成对抗网络（GAN）框架下利用SLM表征完成判别任务，具体面向语音转换场景。我们以StarGANv2-VC为基础，在基于梅尔频谱的判别器之上加入全新的基于SLM的WavLM判别器，同时搭配新设计的SLM特征匹配损失函数，构建出训练阶段无需文本标签的无监督零样本语音转换系统。主观评估结果显示，SLMGAN在自然度方面优于现有最先进的零样本语音转换模型，且相似度表现相当，凸显了基于SLM的判别器在相关应用中的潜力。
英文摘要：
In recent years, large-scale pre-trained speech language models (SLMs) have demonstrated remarkable advancements in various generative speech modeling applications, such as text-to-speech synthesis, voice conversion, and speech enhancement. These applications typically involve mapping text or speech inputs to pre-trained SLM representations, from which target speech is decoded. This paper introduces a new approach, SLMGAN, to leverage SLM representations for discriminative tasks within the generative adversarial network (GAN) framework, specifically for voice conversion. Building upon StarGANv2-VC, we add our novel SLM-based WavLM discriminators on top of the mel-based discriminators along with our newly designed SLM feature matching loss function, resulting in an unsupervised zero-shot voice conversion system that does not require text labels during training. Subjective evaluation results show that SLMGAN outperforms existing state-of-the-art zero-shot voice conversion models in terms of naturalness and achieves comparable similarity, highlighting the potential of SLM-based discriminators for related applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
