arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2606.19792cs.SD
收藏

探索预训练在语音合成中通过微调对音素添加的益处

Exploring Pre-training Benefits on Phoneme Addition through Fine-tuning in Speech Synthesis

CyberAgent, Japan(日本CyberAgent公司);Nagoya University, Japan(日本名古屋大学)

Masato Murata, Koichi Miyazaki, Tomoki Koriyama, Tomoki Toda

AI总结:

研究预训练模型在微调过程中添加新音素时的表现,发现预训练主要提升自然度,但对新音素添加的益处有限。

摘要:

迁移学习广泛用于低资源文本到语音合成。当目标语料包含预训练中未见过的音素时,模型必须在微调期间扩展其音素库存;我们称此过程为“音素添加”。然而,尚不清楚预训练生成已见音素的能力是否有助于此过程。本研究在两个设置中调查音素添加:(1)使用LLM生成的音素控制语料库的模拟设置,可以在不考虑混杂因素的情况下进行研究,以及(2)真实语音跨语言迁移设置(英语到日语),以验证发现是否在实践中成立。两个设置中的实验表明,虽然微调比从头训练实现了更高的自然度,但需要相同或更多的数据才能达到与新音素相当的PER。这些结果表明,预训练主要有助于自然度提升,但对音素添加的益处有限。

英文摘要:

Transfer learning is widely used for low-resource text-to-speech. When the target corpus contains phonemes unseen in pre-training, the model must expand its phoneme inventory during fine-tuning; we call the process "phoneme addition." However, it remains unclear whether the pre-trained ability to generate seen phonemes contributes to this process. This study investigates phoneme addition in two settings: (1) a simulation setup using LLM-generated phoneme-controlled corpora that enables investigation without considering confounding factors, and (2) a real-speech cross-lingual transfer setup (English to Japanese) to validate whether the findings hold in practice. Experiments in both settings showed that while fine-tuning achieved higher naturalness than training from scratch, it required as much or more data to achieve comparable PER for new phonemes. These results indicate that pre-training mainly contributes to naturalness improvement, but offers limited benefit for phoneme addition.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑