解锁基于提示的文本转语音模型中的细粒度和句内说话风格控制
Unlocking Fine-Grained and Within-Utterance Speaking Style Control in Prompt-Based Text-to-Speech Models
AI总结：
针对基于提示的TTS模型缺乏细粒度控制和句内风格变化的问题，提出句间风格插值和句内风格过渡技术，通过嵌入空间方向向量插值和KV缓存交换及滑动窗口注意力掩码实现平滑风格控制。
摘要：
虽然基于提示的文本转语音（TTS）模型支持自然语言驱动的说话风格控制，但它们通常提供有限的细粒度控制，并在整个话语中应用单一的全局风格。这限制了需要跨话语连续风格属性插值和单个话语内时变风格过渡的实际用例。在本文中，我们提出了在现有基于提示的TTS模型中实现这两种能力的新技术。对于句间风格插值，我们计算嵌入空间中对比风格提示之间的方向向量并进行简单插值，从而实现风格特征之间的平滑过渡。对于句内风格过渡，我们首先识别出自回归TTS解码器中对早期标记的强烈注意力偏差，导致初始音频实现主导后续生成。为了减轻这种影响，我们引入了KV缓存交换和滑动窗口注意力掩码。实验表明，我们提出的句间插值在性别转换中实现了99-100%的成功率，高达36 Hz的音高变化，以及高达1.6音节/秒的速度变化。我们的句内过渡保持了0.81-0.91的说话人相似度，并获得了3.48-4.48的感知平滑度分数。
英文摘要：
While prompt-based text-to-speech (TTS) models enable natural language-driven speaking style control, they often provide limited fine-grained control and apply a single global style across an utterance. This restricts practical use cases that require continuous style attribute interpolation across utterances and time-varying style transitions within a single utterance. In this paper, we propose novel techniques to achieve both capabilities in existing prompt-based TTS models. For inter-utterance style interpolation, we compute direction vectors between contrastive style prompts in the embedding space and perform simple interpolation, enabling smooth transitions between style characteristics. For intra-utterance style transition, we first identify a strong attention bias toward early tokens in autoregressive TTS decoders, causing the initial audio realization to dominate subsequent generation. To mitigate this effect, we introduce KV-cache swapping and sliding-window attention masking. Experiments demonstrate that our proposed inter-utterance interpolation achieves a 99-100% success rate in gender conversion, up to 36 Hz pitch variation, and up to 1.6 syllables-per-second speed change. Our intra-utterance transition maintains a speaker similarity of 0.81-0.91 and achieves perceptual smoothness scores of 3.48-4.48.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
