基于流式语音转换的跨语言知识蒸馏用于鲁棒的多语种文本转语音
Cross-lingual Knowledge Distillation via Flow-based Voice Conversion for Robust Polyglot Text-To-Speech
AI总结：
本文提出一种由上游VC模型和下游TTS模型组成的4阶段跨语言语音合成框架，通过流式语音转换进行知识蒸馏，在低资源等设置下优于训练大型多语言TTS模型的最先进方法。
摘要：
在这项工作中，我们引入了一个跨语言语音合成框架，该框架包含上游的Voice Conversion (VC)模型和下游的Text-To-Speech (TTS)模型。提出的框架由4个阶段组成。在前两个阶段，我们使用VC模型将目标语言的语音转换为目标说话人的声音。在第三阶段，转换后的数据与目标语言录音中的语言特征和时长相结合，随后用于训练单说话人声学模型。最后，最后一个阶段涉及训练与语言无关的声码器。我们的评估表明，所提出的范式优于基于训练大型多语言TTS模型的最先进方法。此外，我们的实验证明了该方法在不同模型架构、语言、说话人和数据量下的鲁棒性。而且，我们的解决方案在低资源设置中尤为有益。
英文摘要：
In this work, we introduce a framework for cross-lingual speech synthesis, which involves an upstream Voice Conversion (VC) model and a downstream Text-To-Speech (TTS) model. The proposed framework consists of 4 stages. In the first two stages, we use a VC model to convert utterances in the target locale to the voice of the target speaker. In the third stage, the converted data is combined with the linguistic features and durations from recordings in the target language, which are then used to train a single-speaker acoustic model. Finally, the last stage entails the training of a locale-independent vocoder. Our evaluations show that the proposed paradigm outperforms state-of-the-art approaches which are based on training a large multilingual TTS model. In addition, our experiments demonstrate the robustness of our approach with different model architectures, languages, speakers and amounts of data. Moreover, our solution is especially beneficial in low-resource settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
