X-Voice：通过零样本跨语言语音克隆让每个人都能说30种语言
X-Voice: Enabling Everyone to Speak 30 Languages via Zero-Shot Cross-Lingual Voice Cloning
AI总结：
X-Voice通过双阶段训练和多语言语音合成架构，实现零样本跨语言语音克隆，优于现有多语言系统并在性能上接近大规模模型。
摘要：
本文介绍了X-Voice，一个0.4B多语言零样本语音克隆模型，能够克隆任意声音并使每个人能说30种语言。X-Voice在420万小时多语言语料上训练，使用国际音标（IPA）作为统一表示。为消除对提示文本的依赖，我们设计了双阶段训练范式。在第一阶段，通过标准条件流匹配训练建立X-Voice_s1，生成10000小时说话一致的音频片段作为音频提示。在第二阶段，通过这些音频对进行微调，提示文本被遮蔽以推导出X-Voice_s2，从而实现无需音频提示转录的零样本语音克隆。在架构上，我们通过实现语言标识符的双层注入和分类器自由引导的解耦与调度，扩展了F5-TTS。主观和客观评估结果表明，X-Voice优于现有基于流匹配的多语言系统如LEMAS-TTS，并在性能上接近大规模模型如Qwen3-TTS。为了促进研究透明度和社区发展，我们开源了所有相关资源。
英文摘要：
In this paper, we present X-Voice, a 0.4B multilingual zero-shot voice cloning model that clones arbitrary voices and enables everyone to speak 30 languages. X-Voice is trained on a 420K-hour multilingual corpus using the International Phonetic Alphabet (IPA) as a unified representation. To eliminate the reliance on prompt text without complex preprocessing like forced alignment, we design a two-stage training paradigm. In Stage 1, we establish X-Voice$_{\text{s1}}$ through standard conditional flow-matching training and use it to synthesize 10K hours of speaker-consistent segments as audio prompts. In Stage 2, we fine-tune on these audio pairs with prompt text masked to derive X-Voice$_{\text{s2}}$, which enables zero-shot voice cloning without requiring transcripts of audio prompts. Architecturally, we extend F5-TTS by implementing a dual-level injection of language identifiers and decoupling and scheduling of Classifier-Free Guidance to facilitate multilingual speech synthesis. Subjective and objective evaluation results demonstrate that X-Voice outperforms existing flow-matching based multilingual systems like LEMAS-TTS and achieves zero-shot cross-lingual cloning capabilities comparable to billion-scale models such as Qwen3-TTS. To facilitate research transparency and community advancement, we open-source all related resources.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
