LEMAS: 大规模可扩展多语言音频套件与生成语音模型
LEMAS: Large A 150K-Hour Large-scale Extensible Multilingual Audio Suite with Generative Speech Models
AI总结:
LEMAS数据集通过高效流水线构建,包含10种语言15万小时语音,支持多语言生成合成与编辑,提升跨语言稳定性与编辑质量。
摘要:
我们介绍了LEMAS数据集,据我们所知,目前这是最大的开源多语言语音语料库,具有词级时间戳。覆盖超过150,000小时的10种主要语言,LEMAS数据集通过高效的处理流水线构建,确保高质量的数据和注释。为了验证LEMAS数据集在不同生成范式下的有效性,我们在此数据集上训练了两个具有不同架构和任务专精的基准模型。LEMAS-TTS基于非自回归的流匹配框架,利用数据集的庞大规模和语言多样性,实现稳健的零样本多语言合成。我们提出的音调对抗训练和CTC损失缓解了跨语言音调问题,提高了合成稳定性。同时,LEMAS-Edit采用自回归解码器-only架构,将语音编辑视为掩码标记填充任务。通过利用精确的词级对齐来构建训练掩码,并采用自适应解码策略,实现了无缝、平滑边界语音编辑,具有自然的过渡。实验结果表明,训练于LEMAS数据集的模型实现了高质量的合成和编辑性能,证实了数据集的质量。我们设想,这个具有丰富时间戳注释、细粒度多语言语料库将推动未来基于提示的语音生成系统的发展。
英文摘要:
We present the LEMAS-Dataset, which, to our knowledge, is currently the largest open-source multilingual speech corpus with word-level timestamps. Covering over 150,000 hours across 10 major languages, LEMAS-Dataset is constructed via a efficient data processing pipeline that ensures high-quality data and annotations. To validate the effectiveness of LEMAS-Dataset across diverse generative paradigms, we train two benchmark models with distinct architectures and task specializations on this dataset. LEMAS-TTS, built upon a non-autoregressive flow-matching framework, leverages the dataset's massive scale and linguistic diversity to achieve robust zero-shot multilingual synthesis. Our proposed accent-adversarial training and CTC loss mitigate cross-lingual accent issues, enhancing synthesis stability. Complementarily, LEMAS-Edit employs an autoregressive decoder-only architecture that formulates speech editing as a masked token infilling task. By exploiting precise word-level alignments to construct training masks and adopting adaptive decoding strategies, it achieves seamless, smooth-boundary speech editing with natural transitions. Experimental results demonstrate that models trained on LEMAS-Dataset deliver high-quality synthesis and editing performance, confirming the dataset's quality. We envision that this richly timestamp-annotated, fine-grained multilingual corpus will drive future advances in prompt-based speech generation systems.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。