Align2Speak：通过ASR引导的在线偏好优化改进低资源语言的文本转语音
Align2Speak: Improving TTS for Low Resource Languages via ASR-Guided Online Preference Optimization
AI总结：
提出Align2Speak框架，利用GRPO和ASR奖励优化多语言TTS，在低资源语言上超越微调，在高资源语言上优于DPO。
摘要：
开发低资源语言的高质量文本转语音（TTS）系统因配对文本和语音数据的稀缺而具有挑战性。相比之下，由于大规模多语言预训练的努力，此类语言的自动语音识别（ASR）模型通常更容易获得。我们提出了一种基于组相对策略优化（GRPO）的框架，将自回归多语言TTS模型适应到新语言。我们的方法首先通过使用国际音标（IPA）标记训练多语言基线，为TTS合成建立语言无关的基础。接下来，我们在新语言的有限配对数据上微调此模型，以捕捉目标语言的韵律特征。最后，我们应用GRPO，仅使用未配对的文本和说话人提示，由预训练的ASR、说话人验证和音频质量评估模型的多目标奖励引导来优化模型。实验表明，该流程在低资源语言中产生可理解且说话人一致的语音，显著优于仅微调。此外，我们基于GRPO的框架还提高了高资源语言的TTS性能，超越了离线对齐方法如直接偏好优化（DPO），在可理解性、说话人相似性和音频质量方面表现更优。
英文摘要：
Developing high-quality text-to-speech (TTS) systems for low-resource languages is challenging due to the scarcity of paired text and speech data. In contrast, automatic speech recognition (ASR) models for such languages are often more accessible, owing to large-scale multilingual pre-training efforts. We propose a framework based on Group Relative Policy Optimization (GRPO) to adapt an autoregressive, multilingual TTS model to new languages. Our method first establishes a language-agnostic foundation for TTS synthesis by training a multilingual baseline with International Phonetic Alphabet (IPA) tokens. Next, we fine-tune this model on limited paired data of the new languages to capture the target language's prosodic features. Finally, we apply GRPO to optimize the model using only unpaired text and speaker prompts, guided by a multi-objective reward from pretrained ASR, speaker verification, and audio quality estimation models. Experiments demonstrate that this pipeline produces intelligible and speaker-consistent speech in low-resource languages, substantially outperforming fine-tuning alone. Furthermore, our GRPO-based framework also improves TTS performance in high-resource languages, surpassing offline alignment methods such as Direct Preference Optimization (DPO) yielding superior intelligibility, speaker similarity, and audio quality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
