利用离散语义单元提升希伯来语TTS稳定性
Enhancing TTS Stability in Hebrew using Discrete Semantic Units
AI总结：
本研究针对希伯来语等无变音文字TTS的不稳定性问题，提出基于HuBERT离散语义单元的LOTHM方法，简化建模流程并提升稳定性，自然度与说话人相似度达现有水平，且适配英语。
摘要：
本研究提出了一种改进的文本转语音（TTS）生成方法，可显著提升跨语言的采样稳定性，尤其聚焦于希伯来语。通过利用从自监督模型中获取的、具有更高语音相关性的离散语义单元，我们的方法解决了TTS系统中常见的固有不稳定性问题，尤其是在处理希伯来语这类无变音符号文字时。本模型利用HuBERT码生成针对TTS任务优化的离散表示，从而降低了对基于变音符号的文本处理的依赖。这一进展不仅简化了语言建模流程，还因语义单元的解耦特性提升了语音输出的鲁棒性，并展现出可控性。在声码器中加入说话人嵌入，还有助于捕捉说话人独特的嗓音特征，提升合成语音的自然度。我们的实验结果表明，该方法不仅在希伯来语上保持了高性能，还展现出对英语的适应性，凸显了其在普遍提升TTS系统稳定性方面的有效性。我们命名为LOTHM（Language of The Hebrew Man）的方法，在稳定性上优于现有方法，同时在自然度和说话人相似度上达到了与以往方法相当的水平，是未来语音合成应用的理想选择。样本可在我们的页面pages.cs.huji.ac.il/adiyoss-lab/LoTHM查看。
英文摘要：
This study introduces a refined approach to Text-to-Speech (TTS) generation that significantly enhances sampling stability across languages, with a particular focus on Hebrew. By leveraging discrete semantic units with higher phonetic correlation obtained from a self-supervised model, our method addresses the inherent instability often encountered in TTS systems, especially those dealing with non-diacriticized scripts like Hebrew. Utilizing HuBERT codes, our model generates discrete representations that are optimized for TTS tasks, thereby reducing the dependency on diacritic-based text processing. This advancement not only simplifies the language modeling process but also improves the robustness and shows controllability of the speech output due to disentenglement properties of the semantic units. The inclusion of a speaker embedding in the vocoder further aids in capturing the unique vocal characteristics of the speaker, contributing to the naturalness of the synthesized speech. Our experimental results demonstrate that this approach not only maintains high performance in Hebrew but also shows adaptability to English, underscoring its effectiveness in enhancing stability in TTS systems universally. Our method, named LOTHM (Language of The Hebrew Man), outperforms existing methods in terms of stability while achieving naturalness and speaker similarity on par with previous methods, making it a compelling choice for future speech synthesis applications. Samples can be found in our page pages.cs.huji.ac.il/adiyoss-lab/LoTHM .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
