arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.03887cs.SDeess.AS
收藏

采用新型端到端 Transformer 训练的中库尔德语文本转语音合成

Central Kurdish Text-to-Speech Synthesis with Novel End-to-End Transformer Training

Salahaddin University-Erbil(埃尔比勒萨拉赫丁大学);University of Kurdistan Hawler(库尔德斯坦豪勒大学)

Hawraz A. Ahmad, Tarik A. Rashid

AI总结:

针对中库尔德语 Sorani 方言 TTS 资源不足的问题,提出结合 VAE、对抗潜分布对齐和随机时长预测器的端到端 Transformer 模型,实现高质量实时语音合成,MOS 达 3.94。

摘要:

文本转语音(TTS)模型的最新进展旨在将两阶段流程简化为单阶段训练方法。然而,许多单阶段模型在音频质量方面仍然落后,尤其是在处理库尔德语文本和语音时。目前迫切需要改进库尔德语的文本转语音转换,特别是 Sorani 方言;该方言在近期 TTS 进展中相对受到忽视且代表性不足。本研究提出一种端到端 TTS 模型,用于高效生成高质量库尔德语音频。所提方法利用一个为音频波形重建而预训练的变分自编码器(VAE),并通过对抗训练加以增强。这包括在潜变量中将预训练编码器建立的先验分布与文本编码器的后验分布进行对齐。此外,模型加入随机时长预测器,使合成的库尔德语语音具有多样节奏。通过对齐潜分布并集成随机时长预测器,所提方法支持实时生成自然的库尔德语语音音频,并在音高和节奏上提供灵活性。在自定义数据集上采用平均意见得分(MOS)进行的实证评估,通过主观人工评价证实,本方法具有优于一个单阶段系统及其他两阶段系统的性能,其 MOS 为 3.94。

英文摘要:

Recent advancements in text-to-speech (TTS) models have aimed to streamline the two-stage process into a single-stage training approach. However, many single-stage models still lag behind in audio quality, particularly when handling Kurdish text and speech. There is a critical need to enhance text-to-speech conversion for the Kurdish language, particularly for the Sorani dialect, which has been relatively neglected and is underrepresented in recent text-to-speech advancements. This study introduces an end-to-end TTS model for efficiently generating high-quality Kurdish audio. The proposed method leverages a variational autoencoder (VAE) that is pre-trained for audio waveform reconstruction and is augmented by adversarial training. This involves aligning the prior distribution established by the pre-trained encoder with the posterior distribution of the text encoder within latent variables. Additionally, a stochastic duration predictor is incorporated to imbue synthesized Kurdish speech with diverse rhythms. By aligning latent distributions and integrating the stochastic duration predictor, the proposed method facilitates the real-time generation of natural Kurdish speech audio, offering flexibility in pitches and rhythms. Empirical evaluation via the mean opinion score (MOS) on a custom dataset confirms the superior performance of our approach (MOS of 3.94) compared with that of a one-stage system and other two-staged systems as assessed through a subjective human evaluation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑