THU-HCSI团队面向LIMMITS'24挑战赛的多说话人多语言少样本语音克隆系统
The THU-HCSI Multi-Speaker Multi-Lingual Few-Shot Voice Cloning System for LIMMITS'24 Challenge
AI总结：
本文提出THU-HCSI团队面向LIMMITS'24挑战赛的语音克隆系统，基于YourTTS引入说话人感知编码器与流解码器，结合去噪和平衡采样微调，在track 1取得最佳4.25说话人相似度MOS。
摘要：
本文介绍了由THU-HCSI团队为LIMMITS'24挑战赛开发的多说话人多语言少样本语音克隆系统。为在单语和跨语场景中实现高说话人相似度与自然度，我们基于YourTTS构建系统并进行了若干增强。为进一步提升说话人相似度与语音质量，我们引入了说话人感知文本编码器和基于流且带有Transformer块的解码器。此外，我们对少样本数据进行去噪，将其与预训练数据混合，并采用说话人平衡采样策略，以确保对目标说话人进行有效微调。track 1的官方评估表明，系统取得了4.25的最佳说话人相似度MOS，并获得3.97的自然度MOS。
英文摘要：
This paper presents the multi-speaker multi-lingual few-shot voice cloning system developed by THU-HCSI team for LIMMITS'24 Challenge. To achieve high speaker similarity and naturalness in both mono-lingual and cross-lingual scenarios, we build the system upon YourTTS and add several enhancements. For further improving speaker similarity and speech quality, we introduce speaker-aware text encoder and flow-based decoder with Transformer blocks. In addition, we denoise the few-shot data, mix up them with pre-training data, and adopt a speaker-balanced sampling strategy to guarantee effective fine-tuning for target speakers. The official evaluations in track 1 show that our system achieves the best speaker similarity MOS of 4.25 and obtains considerable naturalness MOS of 3.97.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
