面向大语言模型文本到语音的群体相对策略优化
Group Relative Policy Optimization for Text-to-Speech with Large Language Models
AI总结：
本文提出利用现成ASR模型提供奖励信号，结合CER与NLL构建复合奖励函数，通过GRPO微调基于大语言模型的TTS模型，在零样本场景下显著提升合成语音的可懂度与自然度。
摘要：
本文提出了一种基于群体相对策略优化（GRPO）的方法，通过从现成的自动语音识别（ASR）模型获取奖励信号，来提升基于大语言模型（LLM）的文本到语音（TTS）模型的性能。与以往用于基于LLM的TTS的强化学习方法相比，本方法无需专门的模型来进行奖励计算或训练。此外，我们设计了一种复合奖励函数，将字符错误率（CER）与从ASR模型获得的负对数似然（NLL）相结合，从而提供更具信息量和更准确的奖励信号。我们将GRPO微调应用于预训练的基于LLM的TTS模型，并评估其零样本TTS性能。实验结果表明，所提方法显著提升了合成语音的可懂度和自然度。消融研究和进一步分析证实了整合这两个奖励分量的有效性。
英文摘要：
This paper proposes a GRPO-based approach to enhance the performance of large language model (LLM)-based text-to-speech (TTS) models by deriving rewards from an off-the-shelf automatic speech recognition (ASR) model. Compared to previous reinforcement learning methods for LLM-based TTS, our method requires no dedicated model for reward computation or training. Moreover, we design a composite reward function that combines character error rate (CER) with negative log-likelihood (NLL) obtained from the ASR model, providing more informative and accurate reward signals. We apply GRPO fine-tuning to pre-trained LLM-based TTS models and evaluate their zero-shot TTS performance. Experimental results show that the proposed method substantially improves both the intelligibility and naturalness of synthesized speech. Ablation studies and further analyses confirm the effectiveness of integrating the two reward components.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
