MeanVoiceFlow: 一步非并行语音转换基于均值流
MeanVoiceFlow: One-step Nonparallel Voice Conversion with Mean Flows
AI总结:
MeanVoiceFlow是一种基于均值流的一步非并行语音转换模型,通过结构边缘重建损失和条件扩散输入训练,实现从头开始训练的高效语音转换。
摘要:
在语音转换(VC)应用中,扩散和流匹配模型在语音质量和说话人相似性表现上表现出色。然而,由于其迭代推理,它们受到转换速度慢的限制。因此,我们提出了MeanVoiceFlow,一种基于均值流的新型一步非并行VC模型,可以从头开始训练而不需要预训练或蒸馏。与传统流匹配使用瞬时速度不同,均值流使用平均速度以更准确地计算推理路径上的时间积分。然而,训练平均速度需要其导数来计算目标速度,这可能导致不稳定。因此,我们引入了结构边缘重建损失作为零输入约束,这可以适度正则化模型的输入输出行为,而不会产生有害的统计平均。此外,我们提出了条件扩散输入训练,在训练和推理过程中使用噪声和源数据的混合作为模型的输入。这使模型能够有效利用源信息,同时保持训练和推理之间的一致性。实验结果验证了这些技术的有效性,并证明了MeanVoiceFlow在从头开始训练时的表现与先前的多步和蒸馏模型相当。音频样本可在https://www.kecl.ntt.co.jp/people/kaneko.takuhiro/projects/meanvoiceflow/上获取。
英文摘要:
In voice conversion (VC) applications, diffusion and flow-matching models have exhibited exceptional speech quality and speaker similarity performances. However, they are limited by slow conversion owing to their iterative inference. Consequently, we propose MeanVoiceFlow, a novel one-step nonparallel VC model based on mean flows, which can be trained from scratch without requiring pretraining or distillation. Unlike conventional flow matching that uses instantaneous velocity, mean flows employ average velocity to more accurately compute the time integral along the inference path in a single step. However, training the average velocity requires its derivative to compute the target velocity, which can cause instability. Therefore, we introduce a structural margin reconstruction loss as a zero-input constraint, which moderately regularizes the input-output behavior of the model without harmful statistical averaging. Furthermore, we propose conditional diffused-input training in which a mixture of noise and source data is used as input to the model during both training and inference. This enables the model to effectively leverage source information while maintaining consistency between training and inference. Experimental results validate the effectiveness of these techniques and demonstrate that MeanVoiceFlow achieves performance comparable to that of previous multi-step and distillation-based models, even when trained from scratch. Audio samples are available at https://www.kecl.ntt.co.jp/people/kaneko.takuhiro/projects/meanvoiceflow/.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。