MulliVC：基于循环一致性的多语言语音转换
MulliVC: Multi-lingual Voice Conversion With Cycle Consistency
AI总结：
针对多语言语音转换中语言差异大、同一说话人成对数据稀缺的问题，提出无需成对数据的 MulliVC 三步循环一致性训练系统，实现音色转换并保留内容与源语言韵律。
摘要：
语音转换旨在修改源说话人的声音，使其听起来像目标说话人，同时保留原始语音内容。尽管近年来语音转换取得了显著进展，但多语言语音转换（包括单语言和跨语言两种场景）尚未得到广泛研究。它面临两大主要挑战：1）不同语言之间在韵律和发音习惯上存在相当大的差异；2）缺乏来自同一说话人的成对多语言数据集。本文提出 MulliVC，这是一种新颖的语音转换系统，仅转换音色，并保留原始内容和源语言韵律，且无需多语言成对数据。具体而言，MulliVC 的每个训练步骤包含三个子步骤：第一步使用单语言语音数据训练模型；随后，第二步和第三步从回译中获得启发，构建循环过程，在缺少同一说话人多语言数据的情况下，将音色与其他信息（内容、韵律及其他语言相关信息）解耦。客观和主观结果均表明，MulliVC 在单语言和跨语言情境下都显著优于其他方法，证明了该系统的有效性以及带循环一致性的三步方法的可行性。音频样本可在演示页面（mullivc.github.io）获取。
英文摘要：
Voice conversion aims to modify the source speaker's voice to resemble the target speaker while preserving the original speech content. Despite notable advancements in voice conversion these days, multi-lingual voice conversion (including both monolingual and cross-lingual scenarios) has yet to be extensively studied. It faces two main challenges: 1) the considerable variability in prosody and articulation habits across languages; and 2) the rarity of paired multi-lingual datasets from the same speaker. In this paper, we propose MulliVC, a novel voice conversion system that only converts timbre and keeps original content and source language prosody without multi-lingual paired data. Specifically, each training step of MulliVC contains three substeps: In step one the model is trained with monolingual speech data; then, steps two and three take inspiration from back translation, construct a cyclical process to disentangle the timbre and other information (content, prosody, and other language-related information) in the absence of multi-lingual data from the same speaker. Both objective and subjective results indicate that MulliVC significantly surpasses other methods in both monolingual and cross-lingual contexts, demonstrating the system's efficacy and the viability of the three-step approach with cycle consistency. Audio samples can be found on our demo page (mullivc.github.io).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
