基于LLM的TTS中的动态韵律预测以提高说话人相似度
Dynamic Prosody Prediction in LLM-based TTS for Improving Speaker Similarity
AI总结：
针对LLM-based TTS忽略风格特定韵律模式导致说话人相似度不足的问题，提出基于先前预测语音的动态音节韵律预测方法，显著提升韵律学习能力和说话人相似度。
摘要：
个性化文本到语音（TTS）旨在合成语音中克隆目标说话人，模仿其声音和说话风格。当前基于大型语言模型（LLM）的TTS方法忽略了生成语音中风格特定的韵律模式，导致风格学习不足，从而限制了合成语音中的说话人相似度。为此，我们研究了基于合成语音的韵律学习，并提出基于先前预测语音来预测当前音节的韵律。在三个数据集上获得的实验结果表明，所提出的动态韵律预测方法在增强韵律学习能力方面有效，从而提高了生成语音的说话人相似度。音频样本可在 https://muzw.github.io/dynapros/ 获取。
英文摘要：
Personalized text-to-speech (TTS) aims to clone the target speaker in the synthesized speech, imitating both the voice and speaking style. Current large language model (LLM)-based TTS methods ignore the style-specific prosodic patterns in generated speech, resulting in deficient style learning and thus limiting speaker similarity in synthesized speech. To this end, we investigate the prosody learning conditioned on the synthesized speech, and propose to predict the prosody of the current syllable based on previously predicted speech. Experimental results obtained on three datasets demonstrated the efficacy of the proposed dynamic prosody prediction method in enhancing the prosody learning capability, thereby improving the speaker similarity of the generated speech. Audio samples are available at https://muzw.github.io/dynapros/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
