在任意到一(Any-to-One)语音转换中利用跨语言微调增强多语言语音
Enhancing Polyglot Voices by Leveraging Cross-Lingual Fine-Tuning in Any-to-One Voice Conversion
AI总结:
本文提出一种基于自监督学习的跨语言任意到一语音转换系统及微调策略,无需目标说话人多语言数据即可保留源口音,显著提升了多语言语音的可懂度与整体质量。
摘要:
尽管近年来取得了相当大的进展,但创建人工多语言语音仍然是一项具有挑战性的任务。本文研究了用于语音转换的自监督学习,以创建听起来像母语的多语言语音。我们引入了一种新颖的跨语言任意到一语音转换系统,该系统能够保留源口音,而无需来自目标说话人的多语言数据。此外,我们展示了一种新颖的跨语言微调策略,该策略进一步改善了口音并减少了训练数据需求。使用英语、西班牙语、法语和中文普通话进行的客观和主观评估证实,我们的方法改进了最先进的方法,增强了转换语音的语音可懂度和整体质量,尤其是在跨语言场景中。音频样本可在 https://giuseppe-ruggiero.github.io/a2o-vc-demo/ 获取。
英文摘要:
The creation of artificial polyglot voices remains a challenging task, despite considerable progress in recent years. This paper investigates self-supervised learning for voice conversion to create native-sounding polyglot voices. We introduce a novel cross-lingual any-to-one voice conversion system that is able to preserve the source accent without the need for multilingual data from the target speaker. In addition, we show a novel cross-lingual fine-tuning strategy that further improves the accent and reduces the training data requirements. Objective and subjective evaluations with English, Spanish, French and Mandarin Chinese confirm that our approach improves on state-of-the-art methods, enhancing the speech intelligibility and overall quality of the converted speech, especially in cross-lingual scenarios. Audio samples are available at https://giuseppe-ruggiero.github.io/a2o-vc-demo/
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。