基于Fastpitch的迁移学习改进儿童语音合成
Improved Child Text-to-Speech Synthesis through Fastpitch-based Transfer Learning
AI总结：
针对儿童语音合成的独特挑战，本研究提出基于Fastpitch的迁移学习方法，微调多说话人TTS模型适配儿童语音，经多维度客观评估验证了合成语音的质量，并开源相关数据与代码。
摘要：
近年来，语音合成技术取得了显著进展，能够生成自然且富有表现力的合成语音。其中一个备受关注的领域是合成儿童语音的生成，由于儿童独特的发声特征和发育阶段，该领域面临着独特的挑战。本文提出了一种新颖的方法，利用Fastpitch文本转语音（TTS）模型生成高质量的合成儿童语音。本研究采用迁移学习训练流程，该方法涉及对多说话人TTS模型进行微调以适配儿童语音。我们使用公开的MyST数据集的清理版本（55小时）开展微调实验。我们还发布了本研究生成的合成语音样本原型数据集以及模型代码，以支持后续研究。通过预训练的MOSNet，我们进行了客观评估，结果显示真实儿童语音与合成儿童语音之间存在显著相关性。此外，为验证生成语音的可懂度，我们采用自动语音识别（ASR）模型比较了真实儿童语音与合成儿童语音的词错误率（WER）。我们还使用预训练的说话人编码器测量了真实语音与生成语音之间的说话人相似度。
英文摘要：
Speech synthesis technology has witnessed significant advancements in recent years, enabling the creation of natural and expressive synthetic speech. One area of particular interest is the generation of synthetic child speech, which presents unique challenges due to children's distinct vocal characteristics and developmental stages. This paper presents a novel approach that leverages the Fastpitch text-to-speech (TTS) model for generating high-quality synthetic child speech. This study uses the transfer learning training pipeline. The approach involved finetuning a multi-speaker TTS model to work with child speech. We use the cleaned version of the publicly available MyST dataset (55 hours) for our finetuning experiments. We also release a prototype dataset of synthetic speech samples generated from this research together with model code to support further research. By using a pretrained MOSNet, we conducted an objective assessment that showed a significant correlation between real and synthetic child voices. Additionally, to validate the intelligibility of the generated speech, we employed an automatic speech recognition (ASR) model to compare the word error rates (WER) of real and synthetic child voices. The speaker similarity between the real and generated speech is also measured using a pretrained speaker encoder.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
