Stable-TTS:基于韵律提示的稳定说话人自适应文本到语音合成
Stable-TTS: Stable Speaker-Adaptive Text-to-Speech Synthesis via Prosody Prompting
AI总结:
Stable-TTS提出利用高质量先验样本的韵律提示和先验保持损失,在有限且含噪的目标语音样本条件下实现稳定的说话人自适应语音合成。
摘要:
说话人自适应文本到语音(TTS)合成因其在个性化语音助手服务等领域的广泛应用而受到大量关注。尽管已有多种方法被提出,但它们通常对目标语音样本的数量或质量表现出较高的敏感性。为解决这些局限,我们提出Stable-TTS,一种新颖的说话人自适应TTS框架,该框架利用高质量预训练数据集中的一小部分子集,称为先验样本。具体而言,Stable-TTS通过利用先验样本的高质量韵律实现韵律一致性,同时有效捕捉目标说话人的音色。此外,它在微调过程中采用先验保持损失,以维持对先验样本的合成能力,防止对目标样本过拟合。大量实验表明,即使在目标语音样本数量有限且存在噪声的情况下,Stable-TTS仍具有有效性。
英文摘要:
Speaker-adaptive Text-to-Speech (TTS) synthesis has attracted considerable attention due to its broad range of applications, such as personalized voice assistant services. While several approaches have been proposed, they often exhibit high sensitivity to either the quantity or the quality of target speech samples. To address these limitations, we introduce Stable-TTS, a novel speaker-adaptive TTS framework that leverages a small subset of a high-quality pre-training dataset, referred to as prior samples. Specifically, Stable-TTS achieves prosody consistency by leveraging the high-quality prosody of prior samples, while effectively capturing the timbre of the target speaker. Additionally, it employs a prior-preservation loss during fine-tuning to maintain the synthesis ability for prior samples to prevent overfitting on target samples. Extensive experiments demonstrate the effectiveness of Stable-TTS even under limited amounts of and noisy target speech samples.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。