FlowTTS-GRPO：基于流匹配的文本转语音的多目标奖励优化的在线强化学习
FlowTTS-GRPO: Online Reinforcement Learning with Multi-Objective Reward Optimization for Flow-Matching Based Text-to-Speech
AI总结：
提出FlowTTS-GRPO在线强化学习框架，通过将ODE轨迹转换为SDE路径，直接微调流匹配模型，并采用加权奖励组合、省略无分类器引导等优化，在CosyVoice 3.0和F5-TTS上提升了说话人相似度和感知质量。
摘要：
现有的文本转语音（TTS）强化学习（RL）研究主要集中于大型语言模型（LLM），而流匹配（FM）方法尚未得到充分探索。我们提出了FlowTTS-GRPO，一个用于基于FM的TTS的在线RL框架。通过将常微分方程（ODE）轨迹转换为随机微分方程（SDE）路径，我们的方法能够直接微调开源FM模型，无需辅助模型。我们证明加权奖励组合比概率方案收敛更快，并确定了三种实用优化：训练期间省略无分类器引导（CFG）可加速收敛；合成困难案例可提高鲁棒性；将RL应用于FM组件可改善音频细节指标。在CosyVoice 3.0和F5-TTS上的实验表明，在说话人相似度和感知质量方面获得了客观和主观偏好提升，F5-TTS的可懂度也有所提高。
英文摘要：
Existing Reinforcement Learning (RL) research for Text-to-Speech (TTS) focuses on large language models (LLMs), leaving Flow-Matching (FM) under-explored. We present FlowTTS-GRPO, an online RL framework for FM-based TTS. By converting ordinary differential equation (ODE) trajectories into stochastic differential equation (SDE) paths, our method enables direct fine-tuning of open-source FM models without auxiliary models. We show that a weighted reward combination converges faster than a probabilistic scheme, and identify three practical optimizations: omitting classifier-free guidance (CFG) during training accelerates convergence; synthesizing hard cases improves robustness; and applying RL to the FM component enhances audio-detail metrics. Experiments on CosyVoice 3.0 and F5-TTS demonstrate objective and subjective preference gains in speaker similarity and perceptual quality, with F5-TTS also improving intelligibility.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
