VALL-E R：通过单调对齐实现鲁棒且高效的零样本文本到语音合成
VALL-E R: Robust and Efficient Zero-Shot Text-to-Speech Synthesis via Monotonic Alignment
AI总结：
VALL-E R 在 VALL-E 基础上引入音素单调对齐和编解码器合并策略，在保持语音质量的同时提升零样本 TTS 的鲁棒性与推理效率，推理时间减少超 60%。
摘要：
借助离散神经音频编解码器，大语言模型（LLM）日益被视为零样本文本到语音（TTS）合成的一种有前景的方法。然而，基于采样的解码策略在给生成带来惊人多样性的同时，也带来了诸如错字、遗漏和重复等鲁棒性问题。此外，音频的高采样率也给自回归的推理过程带来了巨大的计算开销。为了解决这些问题，我们提出了 VALL-E R，一个建立在 VALL-E 基础之上的鲁棒且高效的零样本 TTS 系统。具体来说，我们引入了一种音素单调对齐策略，以加强音素与声学序列之间的联系，通过约束声学 token 与其关联的音素相匹配来确保更精确的对齐。此外，我们采用了一种编解码器合并方法，在浅层量化层中对离散编码进行下采样，从而在保持语音输出高质量的同时加快解码速度。受益于这些策略，VALL-E R 获得了对音素的可控性，并通过接近真实文本的词错误率（WER）展示了其强大的鲁棒性。此外，它需要更少的自回归步骤，推理时间减少了超过 60%。这项研究有潜力应用于有意义的项目，包括为失语症患者创建语音。音频样本将在以下网址提供：https://aka.ms/valler。
英文摘要：
With the help of discrete neural audio codecs, large language models (LLM) have increasingly been recognized as a promising methodology for zero-shot Text-to-Speech (TTS) synthesis. However, sampling based decoding strategies bring astonishing diversity to generation, but also pose robustness issues such as typos, omissions and repetition. In addition, the high sampling rate of audio also brings huge computational overhead to the inference process of autoregression. To address these issues, we propose VALL-E R, a robust and efficient zero-shot TTS system, building upon the foundation of VALL-E. Specifically, we introduce a phoneme monotonic alignment strategy to strengthen the connection between phonemes and acoustic sequence, ensuring a more precise alignment by constraining the acoustic tokens to match their associated phonemes. Furthermore, we employ a codec-merging approach to downsample the discrete codes in shallow quantization layer, thereby accelerating the decoding speed while preserving the high quality of speech output. Benefiting from these strategies, VALL-E R obtains controllablity over phonemes and demonstrates its strong robustness by approaching the WER of ground truth. In addition, it requires fewer autoregressive steps, with over 60% time reduction during inference. This research has the potential to be applied to meaningful projects, including the creation of speech for those affected by aphasia. Audio samples will be available at: https://aka.ms/valler.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
