GenerTTS：用于跨语言文本到语音中音色与风格泛化的发音解耦
GenerTTS: Pronunciation Disentanglement for Timbre and Style Generalization in Cross-Lingual Text-to-Speech
AI总结：
本文提出GenerTTS模型，通过基于HuBERT的信息瓶颈解耦音色与发音/风格，并最小化风格与语言的互信息以去除语言特定信息，有效解决了跨语言TTS中的音色和风格泛化挑战，在风格相似度和发音准确性上优于基线系统。
摘要：
跨语言音色与风格可泛化的文本到语音（TTS）旨在合成具有特定参考音色或风格的语音，且该音色或风格从未在目标语言中被训练过。它面临以下挑战：1）音色与发音相关联，因为特定说话者的多语言语音通常难以获取；2）风格与发音相混合，因为语音风格包含语言无关和语言特定的部分。为了应对这些挑战，我们提出了GenerTTS，主要包含以下工作：1）我们精心设计了基于HuBERT的信息瓶颈，以解耦音色与发音/风格；2）我们最小化风格与语言之间的互信息，以丢弃风格嵌入中的语言特定信息。实验表明，GenerTTS在风格相似性和发音准确性方面优于基线系统，并实现了跨语言音色与风格泛化。
英文摘要：
Cross-lingual timbre and style generalizable text-to-speech (TTS) aims to synthesize speech with a specific reference timbre or style that is never trained in the target language. It encounters the following challenges: 1) timbre and pronunciation are correlated since multilingual speech of a specific speaker is usually hard to obtain; 2) style and pronunciation are mixed because the speech style contains language-agnostic and language-specific parts. To address these challenges, we propose GenerTTS, which mainly includes the following works: 1) we elaborately design a HuBERT-based information bottleneck to disentangle timbre and pronunciation/style; 2) we minimize the mutual information between style and language to discard the language-specific information in the style embedding. The experiments indicate that GenerTTS outperforms baseline systems in terms of style similarity and pronunciation accuracy, and enables cross-lingual timbre and style generalization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
