基于风格描述、条件韵律层归一化与扩散GAN的文本到语音合成
Style Description based Text-to-Speech with Conditional Prosodic Layer Normalization based Diffusion GAN
AI总结：
本文提出Prosodic Diff-TTS，通过条件韵律层归一化将风格描述嵌入扩散GAN，在仅4步去噪内实现高保真语音合成，并在多说话人数据集上验证了效果。
摘要：
本文提出一种基于扩散GAN的方法（Prosodic Diff-TTS），以风格描述和内容文本为输入，在仅4个去噪步骤内生成相应的高保真语音。该方法利用新颖的条件韵律层归一化，将风格嵌入融入基于多头注意力的音素编码器和基于梅尔频谱解码器的生成器架构中，以生成语音。风格嵌入通过在音高、语速、情感、性别分类等辅助任务上微调预训练的BERT模型来生成。我们在多说话人LibriTTS和PromptSpeech数据集上验证了所提架构的有效性，使用了多种衡量生成准确率和MOS的定量指标。
英文摘要：
In this paper, we present a Diffusion GAN based approach (Prosodic Diff-TTS) to generate the corresponding high-fidelity speech based on the style description and content text as an input to generate speech samples within only 4 denoising steps. It leverages the novel conditional prosodic layer normalization to incorporate the style embeddings into the multi head attention based phoneme encoder and mel spectrogram decoder based generator architecture to generate the speech. The style embedding is generated by fine tuning the pretrained BERT model on auxiliary tasks such as pitch, speaking speed, emotion,gender classifications. We demonstrate the efficacy of our proposed architecture on multi-speaker LibriTTS and PromptSpeech datasets, using multiple quantitative metrics that measure generated accuracy and MOS.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
