arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2303.09446eess.AScs.AIcs.CLcs.LG
收藏

基于部分输入的可控韵律生成

Controllable Prosody Generation With Partial Inputs

Papercup Ltd.(Papercup公司)

Dan Andrei Iliescu, Devang Savita Ram Mohan, Tian Huey Teh, Zack Hodari

AI总结:

针对文本转语音中韵律难以快速精确控制的问题,提出由部分韵律输入补全并生成完整音频的人在回路框架,仅需约4个输入值即可获得4:1的听者偏好优势。

摘要:

我们研究了文本转语音合成场景中用于生成韵律的人在回路控制问题。控制韵律具有挑战性,因为现有生成模型缺乏可供用户快速、精确修改输出的高效接口。为解决这一问题,我们提出一种新颖框架:用户提供部分输入,由生成模型生成缺失的特征。我们提出一个专门设计的模型,用于编码部分韵律特征并输出完整音频。实验表明,我们的模型展现出人在回路控制机制的两个关键特性:高效性和鲁棒性。即使只使用极少数量的输入值(约4个),该模型也能让用户在听者偏好方面显著提升输出质量,偏好比达到4:1。

英文摘要:

We address the problem of human-in-the-loop control for generating prosody in the context of text-to-speech synthesis. Controlling prosody is challenging because existing generative models lack an efficient interface through which users can modify the output quickly and precisely. To solve this, we introduce a novel framework whereby the user provides partial inputs and the generative model generates the missing features. We propose a model that is specifically designed to encode partial prosodic features and output complete audio. We show empirically that our model displays two essential qualities of a human-in-the-loop control mechanism: efficiency and robustness. With even a very small number of input values (~4), our model enables users to improve the quality of the output significantly in terms of listener preference (4:1).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑