arXiv 2505.079162025-05-12eess.AScs.SD

MiniMax-Speech：采用可学习说话人编码器的固有零样本文本转语音

MiniMax-Speech: Intrinsic Zero-Shot Text-to-Speech with a Learnable Speaker Encoder

MiniMax

Bowen Zhang, Congchao Guo, Geng Yang, Hang Yu, Haozhe Zhang, Heidi Lei, Jialong Mai, Junjie Yan, Kaiyue Yang, Mingqi Yang, Peikai Huang, Ruiyang Jin, Sitan Jian… 展开作者

Bowen Zhang, Congchao Guo, Geng Yang, Hang Yu, Haozhe Zhang, Heidi Lei, Jialong Mai, Junjie Yan, Kaiyue Yang, Mingqi Yang, Peikai Huang, Ruiyang Jin, Sitan Jiang, Weihua Cheng, Yawei Li, Yichen Xiao, Yiying Zhou, Yongmao Zhang, Yuan Lu, Yucen He

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出 MiniMax-Speech，利用可学习说话人编码器和 Flow-VAE 实现零样本高表现力 TTS 与高相似度语音克隆，并支持情感控制、T2V 和 PVC 等扩展。

摘要：

我们介绍 MiniMax-Speech，这是一种基于自回归 Transformer 的文本转语音（TTS）模型，可生成高质量语音。一项关键创新是我们的可学习说话人编码器，它无需参考音频的转写文本即可从中提取音色特征。这使 MiniMax-Speech 能够以零样本方式生成富有表现力、且音色与参考音频一致的语音，同时支持一次性语音克隆，与参考声音具有极高相似度。此外，所提出的 Flow-VAE 提升了合成音频的整体质量。我们的模型支持 32 种语言，并在多项客观和主观评估指标上展现出优异性能。值得注意的是，它在客观语音克隆指标（词错误率和说话人相似度）上取得了最先进（SOTA）结果，并在公开 TTS Arena 排行榜上位居第一。MiniMax-Speech 的另一项关键优势来自说话人编码器提供的鲁棒且解耦的表示：无需修改基础模型即可扩展，从而支持多种应用，例如通过 LoRA 进行任意语音情感控制；通过直接根据文本描述合成音色特征实现文本转声音（T2V）；以及通过使用额外数据微调音色特征实现专业语音克隆（PVC）。我们鼓励读者访问 https://minimax-ai.github.io/tts_tech_report 查看更多示例。

英文摘要：

We introduce MiniMax-Speech, an autoregressive Transformer-based Text-to-Speech (TTS) model that generates high-quality speech. A key innovation is our learnable speaker encoder, which extracts timbre features from a reference audio without requiring its transcription. This enables MiniMax-Speech to produce highly expressive speech with timbre consistent with the reference in a zero-shot manner, while also supporting one-shot voice cloning with exceptionally high similarity to the reference voice. In addition, the overall quality of the synthesized audio is enhanced through the proposed Flow-VAE. Our model supports 32 languages and demonstrates excellent performance across multiple objective and subjective evaluations metrics. Notably, it achieves state-of-the-art (SOTA) results on objective voice cloning metrics (Word Error Rate and Speaker Similarity) and has secured the top position on the public TTS Arena leaderboard. Another key strength of MiniMax-Speech, granted by the robust and disentangled representations from the speaker encoder, is its extensibility without modifying the base model, enabling various applications such as: arbitrary voice emotion control via LoRA; text to voice (T2V) by synthesizing timbre features directly from text description; and professional voice cloning (PVC) by fine-tuning timbre features with additional data. We encourage readers to visit https://minimax-ai.github.io/tts_tech_report for more examples.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。