DSE-TTS：用于跨语言文本到语音的双说话人嵌入
DSE-TTS: Dual Speaker Embedding for Cross-Lingual Text-to-Speech
AI总结：
本文提出DSE-TTS框架，通过双说话人嵌入分别学习语言风格和模仿音色，显著提升了跨语言语音合成的母语感与说话人相似度。
摘要：
尽管单语言语音合成能获得高保真语音，但跨语言文本到语音（CTTS）仍远未令人满意，因为难以准确保留说话人音色（即说话人相似度）并消除第一语言口音（即母语感）。本文证明了向量量化（VQ）声学特征比梅尔频谱图包含更少的说话人信息。基于此发现，我们提出了一种新颖的双说话人嵌入TTS（DSE-TTS）框架，用于具有真实说话风格的CTTS。其中一个嵌入输入声学模型以学习语言说话风格，另一个嵌入则集成到声码器中以模仿目标说话人的音色。实验表明，通过结合两种嵌入，DSE-TTS在跨语言合成中显著优于最先进的SANE-TTS，尤其在母语感方面。
英文摘要：
Although high-fidelity speech can be obtained for intralingual speech synthesis, cross-lingual text-to-speech (CTTS) is still far from satisfactory as it is difficult to accurately retain the speaker timbres(i.e. speaker similarity) and eliminate the accents from their first language(i.e. nativeness). In this paper, we demonstrated that vector-quantized(VQ) acoustic feature contains less speaker information than mel-spectrogram. Based on this finding, we propose a novel dual speaker embedding TTS (DSE-TTS) framework for CTTS with authentic speaking style. Here, one embedding is fed to the acoustic model to learn the linguistic speaking style, while the other one is integrated into the vocoder to mimic the target speaker's timbre. Experiments show that by combining both embeddings, DSE-TTS significantly outperforms the state-of-the-art SANE-TTS in cross-lingual synthesis, especially in terms of nativeness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
