arXiv 2406.178012024-06-22cs.SDcs.CLeess.AS

基于VITS2的多说话人多语言语音克隆系统，面向LIMMITS 2024挑战赛

A multi-speaker multi-lingual voice cloning system based on vits2 for limmits 2024 challenge

School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；School of Information and Communication Engineering, Communication University of China（中国传媒大学信息与通信工程学院）；Shanghai Polytechnic University（上海第二工业大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）

Xiaopeng Wang, Yi Lu, Xin Qi, Zhiyong Wang, Yuankun Xie, Shuchen Shi, Ruibo Fu

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文针对LIMMITS'24挑战赛，基于VITS2架构结合多语言ID与BERT模型，构建了支持七种印度语言的多说话人多语言语音克隆系统，在Track 1和Track 2中分别取得4.02和4.17的说话人相似度得分。

摘要：

本文介绍了为LIMMITS'24 Challenge开发的语音合成系统，主要聚焦于Track 2。该挑战赛的目标是建立一个具备语音克隆能力的多说话人、多语言Indic Text-to-Speech系统，覆盖七种印度语言，并包含男性和女性说话人。该系统使用挑战赛数据进行训练，并针对目标说话人进行few-shot voice cloning微调。评估包括所有七种语言的单语和跨语言合成，并通过主观测试评估自然度与说话人相似度。我们的系统采用VITS2架构，并引入多语言ID和BERT模型以增强上下文语言理解。在不允许使用额外数据的Track 1中，模型Speaker Similarity得分为4.02。在允许使用额外数据的Track 2中，Speaker Similarity得分为4.17。

英文摘要：

This paper presents the development of a speech synthesis system for the LIMMITS'24 Challenge, focusing primarily on Track 2. The objective of the challenge is to establish a multi-speaker, multi-lingual Indic Text-to-Speech system with voice cloning capabilities, covering seven Indian languages with both male and female speakers. The system was trained using challenge data and fine-tuned for few-shot voice cloning on target speakers. Evaluation included both mono-lingual and cross-lingual synthesis across all seven languages, with subjective tests assessing naturalness and speaker similarity. Our system uses the VITS2 architecture, augmented with a multi-lingual ID and a BERT model to enhance contextual language comprehension. In Track 1, where no additional data usage was permitted, our model achieved a Speaker Similarity score of 4.02. In Track 2, which allowed the use of extra data, it attained a Speaker Similarity score of 4.17.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。