基于随机单调对齐的连续自回归建模用于语音合成
Continuous Autoregressive Modeling with Stochastic Monotonic Alignment for Speech Synthesis
AI总结：
该研究提出结合多模态潜在空间VAE与GMM条件自回归模型的语音合成方法，引入随机单调对齐机制，以VALL-E 10.3%的参数实现更优性能，为量化语音语言模型提供高效替代方案。
摘要：
我们提出了一种新颖的语音合成自回归建模方法，将变分自动编码器（VAE，一种具有多模态潜在空间的模型）与使用高斯混合模型（GMM）作为条件概率分布的自回归模型相结合。与依赖残差向量量化的现有方法不同，我们的模型利用VAE潜在空间中的连续语音表示，大幅简化了训练和推理流程。我们还引入随机单调对齐机制以确保严格的单调对齐。在主观和客观评估中，我们的方法显著优于最先进的自回归模型VALL-E，且仅使用VALL-E 10.3%的参数。这证明了连续语音语言模型作为现有量化语音语言模型更高效替代方案的潜力。样本音频可在https://tinyurl.com/gmm-lm-tts获取。
英文摘要：
We propose a novel autoregressive modeling approach for speech synthesis, combining a variational autoencoder (VAE) with a multi-modal latent space and an autoregressive model that uses Gaussian Mixture Models (GMM) as the conditional probability distribution. Unlike previous methods that rely on residual vector quantization, our model leverages continuous speech representations from the VAE's latent space, greatly simplifying the training and inference pipelines. We also introduce a stochastic monotonic alignment mechanism to enforce strict monotonic alignments. Our approach significantly outperforms the state-of-the-art autoregressive model VALL-E in both subjective and objective evaluations, achieving these results with only 10.3\% of VALL-E's parameters. This demonstrates the potential of continuous speech language models as a more efficient alternative to existing quantization-based speech language models. Sample audio can be found at https://tinyurl.com/gmm-lm-tts.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
