DrawSpeech：使用韵律草图作为控制条件的表现力语音合成
DrawSpeech: Expressive Speech Synthesis Using Prosodic Sketches as Control Conditions
AI总结：
针对参考语音难获取、自然语言难以细粒度控制韵律的问题，提出DrawSpeech扩散模型，依据用户绘制的韵律草图恢复音高与能量轮廓并合成语音，实现友好且精确的细粒度韵律控制。
摘要：
控制文本转语音（TTS）系统合成具有用户所期望韵律特征的语音已引起广泛关注。为实现可控性，当前研究主要集中于两个方向：（1）使用参考语音作为韵律提示来引导语音合成；（2）使用自然语言描述来控制生成过程。然而，找到恰好包含用户想要合成的韵律的参考语音需要耗费大量精力。TTS系统中基于描述的引导只能确定整体韵律，难以对合成语音实现细粒度的韵律控制。本文提出DrawSpeech，这是一个以草图为条件的扩散模型，能够根据用户绘制的任意韵律草图生成语音。具体而言，韵律草图被输入DrawSpeech，以提供预期韵律趋势的粗略指示。随后，DrawSpeech基于粗略草图恢复详细的音高和能量轮廓，并合成所需语音。实验结果表明，DrawSpeech能够生成具有多种多样韵律的语音，并能以用户友好的方式精确控制细粒度韵律。我们的实现和音频样本已公开发布。
英文摘要：
Controlling text-to-speech (TTS) systems to synthesize speech with the prosodic characteristics expected by users has attracted much attention. To achieve controllability, current studies focus on two main directions: (1) using reference speech as prosody prompt to guide speech synthesis, and (2) using natural language descriptions to control the generation process. However, finding reference speech that exactly contains the prosody that users want to synthesize takes a lot of effort. Description-based guidance in TTS systems can only determine the overall prosody, which has difficulty in achieving fine-grained prosody control over the synthesized speech. In this paper, we propose DrawSpeech, a sketch-conditioned diffusion model capable of generating speech based on any prosody sketches drawn by users. Specifically, the prosody sketches are fed to DrawSpeech to provide a rough indication of the expected prosody trends. DrawSpeech then recovers the detailed pitch and energy contours based on the coarse sketches and synthesizes the desired speech. Experimental results show that DrawSpeech can generate speech with a wide variety of prosody and can precisely control the fine-grained prosody in a user-friendly manner. Our implementation and audio samples are publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
