DiCLET-TTS：基于扩散模型的跨语言情感迁移用于文本到语音——英语与普通话之间的一项研究
DiCLET-TTS: Diffusion Model based Cross-lingual Emotion Transfer for Text-to-Speech -- A Study between English and Mandarin
AI总结：
本文提出DiCLET-TTS，一种基于扩散模型的跨语言情感迁移方法，通过情感相关语言先验和正交投影解缠模块解决外国口音与情感表现力不足问题，在英语和普通话间实现高效情感迁移。
摘要：
虽然基于单语语料库的跨语言文本到语音（TTS）的性能近年来已显著提升，但生成跨语言语音仍存在外国口音问题，导致自然度受限。此外，当前的跨语言方法忽略了情感建模，而情感是语音传递中不可或缺的副语言信息。在本文中，我们提出了DiCLET-TTS，一种基于扩散模型的跨语言情感迁移方法，能够将情感从源说话人迁移到语内和跨语言的目标说话人。具体而言，为缓解外国口音问题同时提升情感表现力，前向扩散过程的终端分布被参数化为一个与说话人无关但与情感相关的语言先验，该先验由带有情感嵌入作为条件的先验文本编码器生成。为解决情感嵌入中说话人解缠导致的情感表现力较弱的问题，我们提出了一种新颖的基于正交投影的情感解缠模块（OP-EDM），以学习与说话人无关但具有情感判别性的嵌入。此外，引入了一个条件增强的扩散概率模型（DPM）解码器，以增强反向扩散过程中对说话人和情感的建模能力，从而进一步提升语音传递中的情感表现力。跨语言情感迁移实验表明，DiCLET-TTS优于多种竞争模型，并验证了OP-EDM在学习与说话人无关但情感判别性嵌入方面的良好设计。
英文摘要：
While the performance of cross-lingual TTS based on monolingual corpora has been significantly improved recently, generating cross-lingual speech still suffers from the foreign accent problem, leading to limited naturalness. Besides, current cross-lingual methods ignore modeling emotion, which is indispensable paralinguistic information in speech delivery. In this paper, we propose DiCLET-TTS, a Diffusion model based Cross-Lingual Emotion Transfer method that can transfer emotion from a source speaker to the intra- and cross-lingual target speakers. Specifically, to relieve the foreign accent problem while improving the emotion expressiveness, the terminal distribution of the forward diffusion process is parameterized into a speaker-irrelevant but emotion-related linguistic prior by a prior text encoder with the emotion embedding as a condition. To address the weaker emotional expressiveness problem caused by speaker disentanglement in emotion embedding, a novel orthogonal projection based emotion disentangling module (OP-EDM) is proposed to learn the speaker-irrelevant but emotion-discriminative embedding. Moreover, a condition-enhanced DPM decoder is introduced to strengthen the modeling ability of the speaker and the emotion in the reverse diffusion process to further improve emotion expressiveness in speech delivery. Cross-lingual emotion transfer experiments show the superiority of DiCLET-TTS over various competitive models and the good design of OP-EDM in learning speaker-irrelevant but emotion-discriminative embedding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
