IndexTTS：工业级可控高效零样本文本转语音系统
IndexTTS: An Industrial-Level Controllable and Efficient Zero-Shot Text-To-Speech System
AI总结：
针对工业级可控高效零样本TTS需求，本文提出基于XTTS和Tortoise的IndexTTS系统，通过中文字符拼音混合建模、量化方法对比优化及编码器解码器改进，实现发音可控、性能超越XTTS与多款开源主流系统。
摘要：
近年来，基于大语言模型（LLM）的文本转语音（TTS）系统因其高自然度和强大的零样本声音克隆能力逐渐成为工业界主流。本文介绍IndexTTS系统，该系统主要基于XTTS和Tortoise模型，并加入若干新改进：在中文场景中采用字符与拼音结合的混合建模方法，实现多音字和长尾字发音可控；对比分析矢量量化（VQ）与有限标量量化（FSQ）对语音声学token的码本利用率；引入基于Conformer的语音条件编码器，将语音码解码器替换为BigVGAN2。与XTTS相比，IndexTTS在自然度、内容一致性和零样本声音克隆方面显著提升；相较于Fish-Speech、CosyVoice2、FireRedTTS、F5-TTS等开源主流TTS系统，其训练流程更简单、使用更可控、推理速度更快，且性能更优。演示地址：https://index-tts.github.io。
英文摘要：
Recently, large language model (LLM) based text-to-speech (TTS) systems have gradually become the mainstream in the industry due to their high naturalness and powerful zero-shot voice cloning capabilities.Here, we introduce the IndexTTS system, which is mainly based on the XTTS and Tortoise model. We add some novel improvements. Specifically, in Chinese scenarios, we adopt a hybrid modeling method that combines characters and pinyin, making the pronunciations of polyphonic characters and long-tail characters controllable. We also performed a comparative analysis of the Vector Quantization (VQ) with Finite-Scalar Quantization (FSQ) for codebook utilization of acoustic speech tokens. To further enhance the effect and stability of voice cloning, we introduce a conformer-based speech conditional encoder and replace the speechcode decoder with BigVGAN2. Compared with XTTS, it has achieved significant improvements in naturalness, content consistency, and zero-shot voice cloning. As for the popular TTS systems in the open-source, such as Fish-Speech, CosyVoice2, FireRedTTS and F5-TTS, IndexTTS has a relatively simple training process, more controllable usage, and faster inference speed. Moreover, its performance surpasses that of these systems. Our demos are available at https://index-tts.github.io.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
