ED-TTS：基于跨域情感分离的多尺度情感建模用于情感语音合成
ED-TTS: Multi-Scale Emotion Modeling using Cross-Domain Emotion Diarization for Emotional Speech Synthesis
AI总结：
ED-TTS通过结合SER的话语级情感嵌入与SED的帧级情感嵌入，并利用跨域SED预测软标签，实现了多尺度情感语音合成，解决了细粒度情感标注数据稀缺的问题。
摘要：
现有的情感语音合成方法通常利用从参考音频中提取的话语级风格嵌入，忽略了语音韵律固有的多尺度特性。我们提出了ED-TTS，一种多尺度情感语音合成模型，利用语音情感分离（SED）和语音情感识别（SER）在不同层级上建模情感。具体而言，我们提出的方法将SER提取的话语级情感嵌入与从SED获得的细粒度帧级情感嵌入相结合。这些嵌入用于条件化去噪扩散概率模型（DDPM）的逆向过程。此外，我们采用跨域SED来准确预测软标签，以解决监督情感TTS训练中缺乏细粒度情感标注数据集的挑战。
英文摘要：
Existing emotional speech synthesis methods often utilize an utterance-level style embedding extracted from reference audio, neglecting the inherent multi-scale property of speech prosody. We introduce ED-TTS, a multi-scale emotional speech synthesis model that leverages Speech Emotion Diarization (SED) and Speech Emotion Recognition (SER) to model emotions at different levels. Specifically, our proposed approach integrates the utterance-level emotion embedding extracted by SER with fine-grained frame-level emotion embedding obtained from SED. These embeddings are used to condition the reverse process of the denoising diffusion probabilistic model (DDPM). Additionally, we employ cross-domain SED to accurately predict soft labels, addressing the challenge of a scarcity of fine-grained emotion-annotated datasets for supervising emotional TTS training.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
