训练一个紧凑、强大且逼真的文本转语音模型
Training a Compact, Strong, and Realistic Text-to-Speech Model
AI总结：
本研究提出0.6B参数的Audio8-TTS模型，通过三阶段训练结合GRPO与逼真度评判器，实现零样本语音克隆，在可懂度、说话人相似度及感知自然度上达最优性能。
摘要：
文本转语音（TTS）系统通常针对可懂度和说话人相似度进行优化，但这两个目标均未捕捉到区分人类语音表现与合成语音的韵律细节，从而存在逼真度差距，更大的模型和更高质量的数据可缩小但无法消除该差距。由于缺乏直接针对此差距的目标，逼真度仍是TTS领域的持续挑战。我们提出Audio8-TTS，这是一个参数规模为0.6B的紧凑模型，用于零样本语音克隆，在可懂度和说话人相似度方面达到了最先进性能，在感知自然度方面表现尤为突出。为使训练信号与人类感知而非信号级保真度对齐，我们从广泛使用的开源TTS系统构建了质量阶梯，并训练一个逼真度评判器，以针对相同句子的匹配人类录音评估生成的语音。身份擦除目标阻止评判器依赖生成器指纹。我们分三个阶段训练该模型：多语言监督微调；使用词错误率（WER）和说话人相似度（SIM）奖励的组相对策略优化（GRPO）；以及由逼真度评判器引导的GRPO。最后阶段在保持WER性能的同时，提升了感知韵律自然度和说话人相似度。
英文摘要：
Text-to-speech (TTS) systems are conventionally optimized for intelligibility and speaker similarity. However, neither objective captures the prosodic detail that distinguishes human performances from synthetic speech, leaving a realism gap that larger models and higher-quality data can narrow but not close. In the absence of an objective that directly targets this gap, realism remains a persistent challenge in TTS. We present Audio8-TTS, a compact 0.6B-parameter model for zero-shot voice cloning that achieves state-of-the-art performance in intelligibility and speaker similarity, with particularly strong results in perceptual naturalness. To align the training signal with human perception rather than signal-level fidelity, we construct a quality ladder from widely used open-source TTS systems and train a realism critic to assess generated speech against matched human recordings of the same sentences. An identity-erasure objective discourages the critic from relying on generator fingerprints. We train the model in three stages: multilingual supervised fine-tuning; group relative policy optimization (GRPO) with word error rate (WER) and speaker similarity (SIM) rewards; and GRPO guided by the realism critic. The final stage improves perceived prosodic naturalness and speaker similarity while preserving WER performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
