CM-TTS：通过加权采样器与一致性模型提升实时文本到语音合成效率
CM-TTS: Enhancing Real Time Text-to-Speech Synthesis Efficiency through Weighted Samplers and Consistency Models
AI总结：
本文提出基于一致性模型的CM-TTS架构，结合加权采样器实现无对抗训练的高质量实时语音合成，并在实验中优于现有单步语音合成系统。
摘要：
神经文本到语音（TTS）系统在语音助手、电子学习和有声书创作中有着广泛应用。对现代模型（如扩散模型（DMs））的追求有望实现高保真、实时的语音合成。然而，扩散模型中多步采样的效率问题带来了挑战。已有工作尝试将GAN与扩散模型结合，通过近似去噪分布来加速推理，但这会因对抗训练而引入模型收敛问题。为克服这一问题，我们提出了CM-TTS，一种基于一致性模型（CMs）的新型架构。受连续时间扩散模型启发，CM-TTS在无需对抗训练或依赖预训练模型的情况下，以更少的步骤实现高质量语音合成。我们进一步设计了加权采样器，以动态概率将不同采样位置纳入模型训练，确保整个训练过程中的无偏学习。我们提出了一个实时梅尔频谱生成一致性模型，并通过全面评估加以验证。实验结果表明，CM-TTS优于现有的单步语音合成系统，代表了该领域的重要进展。
英文摘要：
Neural Text-to-Speech (TTS) systems find broad applications in voice assistants, e-learning, and audiobook creation. The pursuit of modern models, like Diffusion Models (DMs), holds promise for achieving high-fidelity, real-time speech synthesis. Yet, the efficiency of multi-step sampling in Diffusion Models presents challenges. Efforts have been made to integrate GANs with DMs, speeding up inference by approximating denoising distributions, but this introduces issues with model convergence due to adversarial training. To overcome this, we introduce CM-TTS, a novel architecture grounded in consistency models (CMs). Drawing inspiration from continuous-time diffusion models, CM-TTS achieves top-quality speech synthesis in fewer steps without adversarial training or pre-trained model dependencies. We further design weighted samplers to incorporate different sampling positions into model training with dynamic probabilities, ensuring unbiased learning throughout the entire training process. We present a real-time mel-spectrogram generation consistency model, validated through comprehensive evaluations. Experimental results underscore CM-TTS's superiority over existing single-step speech synthesis systems, representing a significant advancement in the field.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
