Voicebox：大规模文本引导的多语言通用语音生成
Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale
AI总结：
提出Voicebox，一种基于非自回归流匹配的大规模文本引导多语言语音生成模型，通过上下文学习实现零样本TTS等任务，在可懂度和相似度上优于VALL-E且速度提升20倍。
摘要：
GPT和DALL-E等大规模生成模型彻底改变了研究社区。这些模型不仅能生成高保真输出，还是能解决未明确教授任务的通才。相比之下，语音生成模型在规模和任务泛化方面仍处于原始阶段。本文提出Voicebox，这是目前最通用的大规模文本引导语音生成模型。Voicebox是一个非自回归流匹配模型，在超过50K小时未经滤波或增强的语音数据上训练，能在给定音频上下文和文本的情况下进行语音填充。与GPT类似，Voicebox能通过上下文学习执行多种任务，且更灵活，因为它还能以未来上下文为条件。Voicebox可用于单语或跨语言零样本文本转语音合成、噪声去除、内容编辑、风格转换和多样化样本生成。特别地，Voicebox在可懂度（词错率5.9% vs 1.9%）和音频相似度（0.580 vs 0.681）上均优于最先进零样本TTS模型VALL-E，同时速度提升达20倍。
英文摘要：
Large-scale generative models such as GPT and DALL-E have revolutionized the research community. These models not only generate high fidelity outputs, but are also generalists which can solve tasks not explicitly taught. In contrast, speech generative models are still primitive in terms of scale and task generalization. In this paper, we present Voicebox, the most versatile text-guided generative model for speech at scale. Voicebox is a non-autoregressive flow-matching model trained to infill speech, given audio context and text, trained on over 50K hours of speech that are not filtered or enhanced. Similar to GPT, Voicebox can perform many different tasks through in-context learning, but is more flexible as it can also condition on future context. Voicebox can be used for mono or cross-lingual zero-shot text-to-speech synthesis, noise removal, content editing, style conversion, and diverse sample generation. In particular, Voicebox outperforms the state-of-the-art zero-shot TTS model VALL-E on both intelligibility (5.9% vs 1.9% word error rates) and audio similarity (0.580 vs 0.681) while being up to 20 times faster. Audio samples can be found in \url{https://voicebox.metademolab.com}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
