无可验证的语调奖励：迈向基于偏好的语调学习在TTS中
No Verifiable Reward for Prosody: Toward Preference-Guided Prosody Learning in TTS
AI总结：
本文提出一种基于人类偏好的语调优化方法，在KoCC-TTS数据集上实现了更高的语音偏好评分和更优的CER表现，优于传统GRPO方法。
摘要：
最近的工作报告了通过群体相对策略优化（GRPO）在神经文本到语音（TTS）中取得的进步。然而，在缺乏可验证的语调奖励的情况下，GRPO在以转录为导向的信号（CER/NLL）上训练会降低错误率，但使语调崩溃成单调、不自然的语音；添加说话人相似性进一步破坏了训练并降级了CER。我们通过一种迭代的直接偏好优化（DPO）方案来解决这一问题，该方案仅使用每轮几百个人工标注的偏好对来直接优化语调的自然性，同时正则化到当前模型。在KoCC-TTS上，一个经过精心编排的韩国呼叫中心交互真实数据集，捕捉了任务导向的对话，我们的方法达到了最高的人类偏好（ELO）与竞争性的CER，优于GRPO和强大的商业基线。这些结果表明，当语调无法自动奖励时，人类偏好优化提供了一条实用且数据高效的路径来实现自然且稳健的TTS。演示页面可在https://tts.ch.dev上找到。
英文摘要：
Recent work reports gains in neural text-to-speech (TTS) with Group Relative Policy Optimization (GRPO). However, in the absence of a verifiable reward for \textit{prosody}, GRPO trained on transcription-oriented signals (CER/NLL) lowers error rates yet collapses prosody into monotone, unnatural speech; adding speaker-similarity further destabilizes training and degrades CER. We address this with an \textit{iterative Direct Preference Optimization (DPO)} scheme that uses only a few hundred human-labeled preference pairs per round to directly optimize prosodic naturalness while regularizing to the current model. On \textbf{KoCC-TTS}, a curated dataset of authentic Korean call center interactions capturing task-oriented dialogues, our method attains the highest human preference (ELO) with competitive CER, outperforming GRPO and strong commercial baselines. These results suggest that when prosody cannot be rewarded automatically, \textit{human preference optimization} offers a practical and data-efficient path to natural and robust TTS. The demo page is available at \href{https://tts.ch.dev}
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
