SimpleSpeech：迈向基于标量隐变量 Transformer 扩散模型的简单高效文本转语音
SimpleSpeech: Towards Simple and Efficient Text-to-Speech with Scalar Latent Transformer Diffusion Models
AI总结：
提出 SimpleSpeech，结合 SQ-Codec 标量隐空间与 Transformer 扩散模型，在无对齐纯语音数据上实现高质量、快速的非自回归 TTS。
摘要：
在本研究中，我们提出了一个简单高效的、基于扩散的非自回归（Non-Autoregressive，NAR）文本转语音（text-to-speech，TTS）系统，名为 SimpleSpeech。其简单性体现在三个方面：（1）它可以在仅含语音的数据集上训练，无需任何对齐信息；（2）它直接以纯文本作为输入，并通过 NAR 方式生成语音；（3）它尝试在有限且紧凑的隐空间中对语音建模，从而缓解扩散的建模难度。更具体地说，我们提出了一种采用标量量化的新型语音编解码器模型（SQ-Codec），SQ-Codec 能有效地将复杂语音信号映射到一个有限且紧凑的隐空间，该空间被命名为标量隐空间。得益于 SQ-Codec，我们在 SQ-Codec 的标量隐空间中应用了一种新型 transformer 扩散模型。我们在 4000 小时的仅含语音数据集上训练 SimpleSpeech，它展现出自然的韵律和声音克隆能力。与以往的大规模 TTS 模型相比，它在语音质量和生成速度上均表现出显著提升。演示已发布。
英文摘要：
In this study, we propose a simple and efficient Non-Autoregressive (NAR) text-to-speech (TTS) system based on diffusion, named SimpleSpeech. Its simpleness shows in three aspects: (1) It can be trained on the speech-only dataset, without any alignment information; (2) It directly takes plain text as input and generates speech through an NAR way; (3) It tries to model speech in a finite and compact latent space, which alleviates the modeling difficulty of diffusion. More specifically, we propose a novel speech codec model (SQ-Codec) with scalar quantization, SQ-Codec effectively maps the complex speech signal into a finite and compact latent space, named scalar latent space. Benefits from SQ-Codec, we apply a novel transformer diffusion model in the scalar latent space of SQ-Codec. We train SimpleSpeech on 4k hours of a speech-only dataset, it shows natural prosody and voice cloning ability. Compared with previous large-scale TTS models, it presents significant speech quality and generation speed improvement. Demos are released.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
