arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2202.13066eess.AScs.SD
收藏

重新审视文本转语音中的过度平滑

Revisiting Over-Smoothness in Text to Speech

Zhejiang University(浙江大学);Microsoft Research(微软研究院)

Yi Ren, Xu Tan, Tao Qin, Zhou Zhao, Tie-Yan Liu

摘要:

非自回归文本转语音(NAR-TTS)模型由于其生成速度快而受到学术界和工业界的广泛关注。NAR-TTS模型的一个局限性是,它们在生成语音梅尔频谱图时忽略了时域和频域中的相关性,从而导致模糊和过度平滑的结果。在这项工作中,我们从一个新的角度重新审视这个过度平滑问题:过度平滑的程度是由数据分布的复杂性和建模方法的能力之间的差距决定的。简化数据分布和改进建模方法都可以缓解这个问题。因此,我们首先研究降低数据分布复杂性的方法。然后,我们进行了全面的研究,NAR-TTS模型,使用一些先进的建模方法。基于这些研究,我们发现:1)提供额外条件输入的方法降低了建模数据分布的复杂性,从而缓解了过度平滑问题并实现了更好的语音质量。2)在先进的建模方法中,拉普拉斯混合损失在建模多模态分布方面表现良好,并且具有其简单性,而GAN和Glow实现了最佳的语音质量,同时遭受增加的训练或模型复杂性。3)这两类方法可以结合使用,以进一步缓解过度平滑,提高语音质量。4)我们在多说话人数据集上的实验得出了类似的结论,提供更多的方差信息可以降低目标数据分布建模的难度,减轻对模型容量的要求。

英文摘要:

Non-autoregressive text to speech (NAR-TTS) models have attracted much attention from both academia and industry due to their fast generation speed. One limitation of NAR-TTS models is that they ignore the correlation in time and frequency domains while generating speech mel-spectrograms, and thus cause blurry and over-smoothed results. In this work, we revisit this over-smoothing problem from a novel perspective: the degree of over-smoothness is determined by the gap between the complexity of data distributions and the capability of modeling methods. Both simplifying data distributions and improving modeling methods can alleviate the problem. Accordingly, we first study methods reducing the complexity of data distributions. Then we conduct a comprehensive study on NAR-TTS models that use some advanced modeling methods. Based on these studies, we find that 1) methods that provide additional condition inputs reduce the complexity of data distributions to model, thus alleviating the over-smoothing problem and achieving better voice quality. 2) Among advanced modeling methods, Laplacian mixture loss performs well at modeling multimodal distributions and enjoys its simplicity, while GAN and Glow achieve the best voice quality while suffering from increased training or model complexity. 3) The two categories of methods can be combined to further alleviate the over-smoothness and improve the voice quality. 4) Our experiments on the multi-speaker dataset lead to similar conclusions as above and providing more variance information can reduce the difficulty of modeling the target data distribution and alleviate the requirements for model capacity.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑