通过对比学习和扩散模型增强自然语言引导的情感文本转语音可控性
Enhancing Emotional Text-to-Speech Controllability with Natural Language Guidance through Contrastive Learning and Diffusion Models
AI总结：
提出 ParaEVITS 情感 TTS 框架，结合 ParaCLAP 文本-音频编码与扩散模型，使用自然语言描述控制情感渲染并保持语音质量。
摘要：
尽管当前的情感文本转语音（TTS）系统能够生成清晰度很高的情感语音，但对输出语音的情感渲染实现精细控制仍然是一项重大挑战。本文提出 ParaEVITS，这是一种新颖的情感 TTS 框架，利用自然语言的组合性来增强对情感渲染的控制。通过引入受 ParaCLAP 启发的文本-音频编码器（ParaCLAP 是一种面向计算副语言学的对比语言-音频预训练（CLAP）模型），扩散模型被训练为根据文本情感风格描述生成情感嵌入。我们的框架首先使用音频编码器在参考音频上训练，然后微调扩散模型以处理来自 ParaCLAP 文本编码器的文本输入。在推理过程中，仅使用文本条件即可操控音高、抖动和响度等语音属性。实验表明，ParaEVITS 能够有效控制情感渲染，且不会损害语音质量。语音演示已公开发布。
英文摘要：
While current emotional text-to-speech (TTS) systems can generate highly intelligible emotional speech, achieving fine control over emotion rendering of the output speech still remains a significant challenge. In this paper, we introduce ParaEVITS, a novel emotional TTS framework that leverages the compositionality of natural language to enhance control over emotional rendering. By incorporating a text-audio encoder inspired by ParaCLAP, a contrastive language-audio pretraining (CLAP) model for computational paralinguistics, the diffusion model is trained to generate emotional embeddings based on textual emotional style descriptions. Our framework first trains on reference audio using the audio encoder, then fine-tunes a diffusion model to process textual inputs from ParaCLAP's text encoder. During inference, speech attributes such as pitch, jitter, and loudness are manipulated using only textual conditioning. Our experiments demonstrate that ParaEVITS effectively control emotion rendering without compromising speech quality. Speech demos are publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
