arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.14523cs.CLcs.SDeess.AS

Daisy-TTS:通过韵律嵌入分解模拟更广泛的情感谱系

Daisy-TTS: Simulating Wider Spectrum of Emotions via Prosody Embedding Decomposition

MBZUAI(穆罕默德·本·扎耶德人工智能大学)

Rendi Chevi, Alham Fikri Aji

AI总结:

提出基于情感结构模型的情感文本转语音设计 Daisy-TTS,通过韵律编码器学习情感可分离的韵律嵌入,实现了对基本情感、次级情感混合、情感强度及情感极性的模拟,在感知评估中优于基线。

摘要:

我们经常以多层面的方式口头表达情感,这些情感在强度上可能有所不同,并且可能不仅是单一情感,而是混合情感。这种广泛的情感谱系在情感结构模型中得到了充分研究,该模型将各种情感表示为基本情感在不同强度下的衍生产物。在本文中,我们提出了一种情感文本转语音设计,基于结构模型来模拟更广泛的情感谱系。我们提出的设计 Daisy-TTS 结合了韵律编码器,以学习情感可分离的韵律嵌入作为情感的代理。这种情感表示允许模型模拟:(1)从训练样本中学习的基本情感,(2)作为基本情感混合的次级情感,(3)通过缩放情感嵌入实现的强度级别,以及(4)通过否定情感嵌入实现的情感极性。通过一系列感知评估,与基线相比,Daisy-TTS 展现出总体更高的情感语音自然度和情感可感知性。

英文摘要:

We often verbally express emotions in a multifaceted manner, they may vary in their intensities and may be expressed not just as a single but as a mixture of emotions. This wide spectrum of emotions is well-studied in the structural model of emotions, which represents variety of emotions as derivative products of primary emotions with varying degrees of intensity. In this paper, we propose an emotional text-to-speech design to simulate a wider spectrum of emotions grounded on the structural model. Our proposed design, Daisy-TTS, incorporates a prosody encoder to learn emotionally-separable prosody embedding as a proxy for emotion. This emotion representation allows the model to simulate: (1) Primary emotions, as learned from the training samples, (2) Secondary emotions, as a mixture of primary emotions, (3) Intensity-level, by scaling the emotion embedding, and (4) Emotions polarity, by negating the emotion embedding. Through a series of perceptual evaluations, Daisy-TTS demonstrated overall higher emotional speech naturalness and emotion perceiveability compared to the baseline.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑