提示一致性推理用于零样本流匹配文本到语音模型
Prompt-Consistency Inference for Zero-Shot Flow-Matching Text-to-Speech Models
浏览论文内容
中文总结 AI 辅助
针对零样本流匹配TTS中提示状态漂移问题，提出无需训练的提示一致性推理（PCI），在速度评估前恢复提示块解析值，提升说话人相似度和可懂度。
中文摘要 AI 辅助
近年来，流匹配模型在零样本文本到语音合成方面取得了显著进展。这些模型以音频提示和文本为条件，学习一个速度场，并通过迭代求解常微分方程（ODE）来生成语音。在推理过程中，求解器演化一个跨越提示区域和待生成区域的单一状态，尽管最终仅保留生成区域。然而，被丢弃的提示状态仍然重要；其中间值通过耦合两个区域的速度场影响生成过程。随着采样的继续，该状态可能偏离规定的条件路径，从而在后续生成更新中引入差异。与未知的生成轨迹不同，提示路径可以从参考音频和初始噪声中以闭式形式获得。我们利用这一观察提出了提示一致性推理（PCI），这是一种无需训练的规则，在每次速度评估之前将提示块恢复为其解析值，同时保持生成块不变。PCI在不增加额外网络评估的情况下，提高了所评估的流匹配TTS骨干网络的说话人相似度和可懂度。我们的消融研究进一步表明，PCI保持了较小的步后提示差异，并且覆盖后期采样阶段的修正恢复了许多观察到的相似度增益。
英文摘要
In recent years, flow-matching models have produced significant improvements in zero-shot text-to-speech synthesis. Conditioned on an audio prompt and text, these models learn a velocity field and generate speech by iteratively solving an ODE. During inference, the solver evolves a single state spanning both the prompt and the region to be generated, although only the generated region is ultimately retained. The discarded prompt state, however, still matters; its intermediate values influence generation through the velocity field that couples the two regions. As sampling continues, this state can drift away from the prescribed conditional path, introducing a discrepancy into subsequent generation updates. Unlike the unknown generated trajectory, the prompt path is available in closed form from the reference audio and the initial noise. We exploit this observation with Prompt-Consistency Inference (PCI), a training-free rule that restores the prompt block to its analytic value before each velocity evaluation, while leaving the generated block unchanged. PCI improves speaker similarity and intelligibility across the evaluated flow-matching TTS backbones without additional network evaluations. Our ablation studies further show that PCI keeps post-step prompt discrepancies smaller and that corrections covering the later sampling stages recover much of the observed similarity gain.
发表机构
- Microsoft Corporation（微软公司）
机构由 AI 辅助整理，请以论文原文为准。
