IndicVoices-R：解锁大规模多语言多说话人语音语料库以推进印度语TTS规模化发展
IndicVoices-R: Unlocking a Massive Multilingual Multi-speaker Speech Corpus for Scaling Indian TTS
AI总结：
针对印度语言高质量TTS训练数据稀缺的问题，研究团队通过优化低质量ASR数据集构建了规模最大的印度多语言TTS数据集IndicVoices-R，推出对应评估基准，验证了其提升TTS模型零样本说话人泛化能力的效果，并开源资源与全语种TTS模型。
摘要：
文本转语音（TTS）合成领域的最新进展表明，利用海量网络数据训练的大规模模型可生成自然度极高的语音输出。然而，由于LibriVox或YouTube等平台缺乏高质量的人工字幕数据，印度语言的此类训练数据十分稀缺。为填补这一空白，我们对现有包含低质量环境下采集的自然对话的大规模自动语音识别（ASR）数据集进行优化，以生成高质量的TTS训练数据。我们的处理流水线利用了基于英语训练、再应用于印度语言的去噪与语音增强模型的跨语言泛化能力。由此构建出IndicVoices-R（简称IV-R），这是首个源自ASR数据集的最大规模印度多语言TTS数据集，涵盖22种印度语言的10496名说话人，包含1704小时的高质量语音数据。IV-R的语音质量可与LJSpeech、LibriTTS和IndicTTS等金标准TTS数据集相媲美。我们还推出了IV-R Benchmark（IV-R基准测试），这是首个用于评估TTS模型在印度语音上的零样本、少样本和多样本说话人泛化能力的基准，确保了年龄、性别和风格的多样性。实验证明，在高质量IndicTTS数据集与我们的IV-R数据集的组合数据上微调英语预训练模型，相比仅在IndicTTS数据集上微调，能实现更优的零样本说话人泛化效果。此外，我们的评估发现，基于现有数据集训练的TTS模型在印度语音上的零样本泛化能力有限，而通过在包含跨语系多样说话人的我们的数据集上微调模型，这一能力得到了提升。我们已开源所有数据和代码，并发布了首个支持全部22种印度官方语言的TTS模型。
英文摘要：
Recent advancements in text-to-speech (TTS) synthesis show that large-scale models trained with extensive web data produce highly natural-sounding output. However, such data is scarce for Indian languages due to the lack of high-quality, manually subtitled data on platforms like LibriVox or YouTube. To address this gap, we enhance existing large-scale ASR datasets containing natural conversations collected in low-quality environments to generate high-quality TTS training data. Our pipeline leverages the cross-lingual generalization of denoising and speech enhancement models trained on English and applied to Indian languages. This results in IndicVoices-R (IV-R), the largest multilingual Indian TTS dataset derived from an ASR dataset, with 1,704 hours of high-quality speech from 10,496 speakers across 22 Indian languages. IV-R matches the quality of gold-standard TTS datasets like LJSpeech, LibriTTS, and IndicTTS. We also introduce the IV-R Benchmark, the first to assess zero-shot, few-shot, and many-shot speaker generalization capabilities of TTS models on Indian voices, ensuring diversity in age, gender, and style. We demonstrate that fine-tuning an English pre-trained model on a combined dataset of high-quality IndicTTS and our IV-R dataset results in better zero-shot speaker generalization compared to fine-tuning on the IndicTTS dataset alone. Further, our evaluation reveals limited zero-shot generalization for Indian voices in TTS models trained on prior datasets, which we improve by fine-tuning the model on our data containing diverse set of speakers across language families. We open-source all data and code, releasing the first TTS model for all 22 official Indian languages.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
