arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.09311eess.AScs.SD
收藏

通过稳定共振峰生成提升基于扩散的零样本语音合成的鲁棒性

Improving Robustness of Diffusion-Based Zero-Shot Speech Synthesis via Stable Formant Generation

DeepBrain AI

Changjin Han, Seokgi Lee, Gyuhyeon Nam, Gyeongsu Chae

AI总结:

针对扩散模型零样本语音合成中被忽视的发音不稳定问题,提出结合源-滤波器理论的StableForm-TTS框架,通过稳定共振峰生成提升发音准确率与自然度,且具备可扩展性。

摘要:

扩散模型已在文本转语音(TTS)领域取得显著成功,甚至在零样本场景中亦是如此。近期研究致力于解决推理速度与音质之间的权衡问题,这通常被视为扩散模型的主要缺陷。然而,我们发现一个关键的发音错误问题正被忽视。我们的初步研究揭示了扩散过程导致的发音不稳定现象。基于这一观察,我们提出了StableForm-TTS,这是一种新型零样本语音合成框架,旨在在保留扩散建模优势的同时生成鲁棒的发音。通过率先将源-滤波器理论应用于扩散TTS,我们提出了一种用于稳定共振峰生成的精巧架构。在未见过的说话人上的实验结果表明,我们的模型在发音准确率和自然度方面优于当前最优方法,且说话人相似度相当。此外,随着数据量和模型规模的增加,我们的模型展现出有效的可扩展性。音频样本可在线获取:https://deepbrainai-research.github.io/stableformtts/。

英文摘要:

Diffusion models have achieved remarkable success in text-to-speech (TTS), even in zero-shot scenarios. Recent efforts aim to address the trade-off between inference speed and sound quality, often considered the primary drawback of diffusion models. However, we find a critical mispronunciation issue is being overlooked. Our preliminary study reveals the unstable pronunciation resulting from the diffusion process. Based on this observation, we introduce StableForm-TTS, a novel zero-shot speech synthesis framework designed to produce robust pronunciation while maintaining the advantages of diffusion modeling. By pioneering the adoption of source-filter theory in diffusion TTS, we propose an elaborate architecture for stable formant generation. Experimental results on unseen speakers show that our model outperforms the state-of-the-art method in terms of pronunciation accuracy and naturalness, with comparable speaker similarity. Moreover, our model demonstrates effective scalability as both data and model sizes increase. Audio samples are available online: https://deepbrainai-research.github.io/stableformtts/.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑