用于文本到语音合成的语音增强语言建模
Phonetic Enhanced Language Modeling for Text-to-Speech Synthesis
AI总结：
本文提出语音增强语言建模方法，以语音信息丰富的自监督表示作为自回归语言模型训练目标，并用非自回归模型预测离散声学编解码器，从而减少误差传播，提升TTS模型的鲁棒性和性能。
摘要：
近年来基于语言模型的文本到语音（TTS）框架展现出可扩展性和上下文学习能力。然而，由于自回归语言建模过程中语音单元预测误差的累积，它们存在鲁棒性问题。本文提出一种语音增强的语言建模方法来提升TTS模型的性能。我们利用语音信息丰富的自监督表示作为自回归语言模型的训练目标。随后，采用一个非自回归模型来预测包含细粒度声学细节的离散声学编解码器。该TTS模型在自回归训练阶段仅专注于语言建模，从而减少非自回归训练中出现的误差传播。客观和主观评估均验证了所提方法的有效性。
英文摘要：
Recent language model-based text-to-speech (TTS) frameworks demonstrate scalability and in-context learning capabilities. However, they suffer from robustness issues due to the accumulation of errors in speech unit predictions during autoregressive language modeling. In this paper, we propose a phonetic enhanced language modeling method to improve the performance of TTS models. We leverage self-supervised representations that are phonetically rich as the training target for the autoregressive language model. Subsequently, a non-autoregressive model is employed to predict discrete acoustic codecs that contain fine-grained acoustic details. The TTS model focuses solely on linguistic modeling during autoregressive training, thereby reducing the error propagation that occurs in non-autoregressive training. Both objective and subjective evaluations validate the effectiveness of our proposed method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
