arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2311.07965cs.SDeess.AS
收藏

DQR-TTS:基于动态量化表示的半监督语音合成

DQR-TTS: Semi-supervised Text-to-speech Synthesis with Dynamic Quantized Representation

Ping An Technology (Shenzhen) Co., Ltd.(平安科技(深圳)有限公司);University of Science and Technology of China(中国科学技术大学)

Jianzong Wang, Pengcheng Li, Xulong Zhang, Ning Cheng, Jing Xiao

AI总结:

针对低资源下语音合成数据不足的问题,提出带动态量化表示模块的半监督TTS模型DQR-TTS,用非配对数据扩展码本,在120分钟内配对数据下性能优于现有方法。

摘要:

现有大多数基于神经网络的语音合成方法依赖大规模数据集,在低资源条件下面临挑战。本文提出一种新型半监督语音合成模型,可同时利用配对数据和非配对数据学习以解决该问题。模型的核心组件是动态量化表示模块,被集成到序列自编码器中。当输入配对数据时,该模块包含一个可训练码本,在配对数据的监督下学习量化表示。但由于低资源场景下配对数据有限,难以覆盖所有音素,因此训练过程中会输入非配对数据,通过添加与现有量化表示向量差异足够大的向量来扩展动态码本。实验表明,在配对数据不足120分钟的情况下,所提方法在主观和客观指标上均优于现有方法。

英文摘要:

Most existing neural-based text-to-speech methods rely on extensive datasets and face challenges under low-resource condition. In this paper, we introduce a novel semi-supervised text-to-speech synthesis model that learns from both paired and unpaired data to address this challenge. The key component of the proposed model is a dynamic quantized representation module, which is integrated into a sequential autoencoder. When given paired data, the module incorporates a trainable codebook that learns quantized representations under the supervision of the paired data. However, due to the limited paired data in low-resource scenario, these paired data are difficult to cover all phonemes. Then unpaired data is fed to expand the dynamic codebook by adding quantized representation vectors that are sufficiently distant from the existing ones during training. Experiments show that with less than 120 minutes of paired data, the proposed method outperforms existing methods in both subjective and objective metrics.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑