用于 LIMMITS 2023 挑战赛的多说话人多语言 VQTTS 系统
Multi-Speaker Multi-Lingual VQTTS System for LIMMITS 2023 Challenge
AI总结：
SJTU X-LANCE 团队基于 VQTTS 构建多说话人多语言 TTS 系统参加 LIMMITS 2023 挑战赛，通过引入说话人和语言嵌入并采用跨语言嵌入分配策略，在 track 1 自然度主观测试中以 4.77 分夺冠。
摘要：
本文描述了 SJTU X-LANCE 团队为 LIMMITS 2023 挑战赛开发的系统，我们主要关注在 track 1 自然度上获胜的系统。该挑战赛的目标是为 Marathi、Hindi 和 Telugu 构建一个多说话人多语言文本转语音（TTS）系统。在给定的数据集中，每种语言各有一名男性和一名女性说话人。在 track 1 中，每位说话人只能选取 5 小时的数据来训练 TTS 模型。我们的系统基于最近提出的 VQTTS，它利用 VQ 声学特征而不是 mel-spectrogram。我们向 VQTTS 引入了额外的说话人嵌入和语言嵌入，以控制说话人和语言信息。在需要以跨语言说话人的声音合成语音的跨语言评估中，我们向声学模型提供母语说话人的嵌入，并向声码器提供目标说话人的嵌入。在自然度的主观 MOS 听力测试中，我们的系统取得了 4.77 的成绩，排名第一。
英文摘要：
In this paper, we describe the systems developed by the SJTU X-LANCE team for LIMMITS 2023 Challenge, and we mainly focus on the winning system on naturalness for track 1. The aim of this challenge is to build a multi-speaker multi-lingual text-to-speech (TTS) system for Marathi, Hindi and Telugu. Each of the languages has a male and a female speaker in the given dataset. In track 1, only 5 hours data from each speaker can be selected to train the TTS model. Our system is based on the recently proposed VQTTS that utilizes VQ acoustic feature rather than mel-spectrogram. We introduce additional speaker embeddings and language embeddings to VQTTS for controlling the speaker and language information. In the cross-lingual evaluations where we need to synthesize speech in a cross-lingual speaker's voice, we provide a native speaker's embedding to the acoustic model and the target speaker's embedding to the vocoder. In the subjective MOS listening test on naturalness, our system achieves 4.77 which ranks first.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
