MeanVC: 一种轻量级和流式零样本语音转换方法
MeanVC: Lightweight and Streaming Zero-Shot Voice Conversion via Mean Flows
AI总结：
MeanVC通过引入均值流和扩散对抗后训练，实现轻量级、高效的零样本语音转换，提升语音质量和效率。
摘要：
零样本语音转换（VC）旨在将源说话人的音色转移到任何未见过的目标说话人，同时保持语言内容。日益增长的应用场景要求模型具有流式推理能力。这催生了对同时快速、轻量级和高保真的模型的迫切需求。然而，现有的流式方法通常依赖于自回归（AR）或非自回归（NAR）框架，这些框架要么需要大参数规模才能实现强性能，要么难以泛化到未见过的说话人。在本研究中，我们提出了MeanVC，一种轻量级和流式零样本VC方法。MeanVC引入了具有分块自回归去噪策略的扩散变换器，结合了AR和NAR范式的优势，实现了高效的流式处理。通过引入均值流，MeanVC在训练过程中回归平均速度场，使得在单次采样步骤中，通过直接从流轨迹的起点到终点进行映射，实现零样本VC，具有优越的语音质量和说话人相似性。此外，我们还结合了扩散对抗后训练以减轻过平滑问题，并进一步提升语音质量。实验结果表明，MeanVC显著优于现有的零样本流式VC系统，在实现更高质量的转换质量的同时，具有更高的效率和显著更少的参数。音频演示和代码可在https://aslp-lab.github.io/MeanVC上公开获取。
英文摘要：
Zero-shot voice conversion (VC) aims to transfer timbre from a source speaker to any unseen target speaker while preserving linguistic content. Growing application scenarios demand models with streaming inference capabilities. This has created a pressing need for models that are simultaneously fast, lightweight, and high-fidelity. However, existing streaming methods typically rely on either autoregressive (AR) or non-autoregressive (NAR) frameworks, which either require large parameter sizes to achieve strong performance or struggle to generalize to unseen speakers. In this study, we propose MeanVC, a lightweight and streaming zero-shot VC approach. MeanVC introduces a diffusion transformer with a chunk-wise autoregressive denoising strategy, combining the strengths of both AR and NAR paradigms for efficient streaming processing. By introducing mean flows, MeanVC regresses the average velocity field during training, enabling zero-shot VC with superior speech quality and speaker similarity in a single sampling step by directly mapping from the start to the endpoint of the flow trajectory. Additionally, we incorporate diffusion adversarial post-training to mitigate over-smoothing and further enhance speech quality. Experimental results demonstrate that MeanVC significantly outperforms existing zero-shot streaming VC systems, achieving superior conversion quality with higher efficiency and significantly fewer parameters. Audio demos and code are publicly available at https://aslp-lab.github.io/MeanVC.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
