StableVC：基于条件流匹配的风格可控零样本语音转换
StableVC: Style Controllable Zero-Shot Voice Conversion with Conditional Flow Matching
AI总结：
本文提出StableVC，通过分解语言内容、音色与风格，并结合条件流匹配和自适应门控双重注意力，实现对不同未见说话人音色与风格的独立可控零样本语音转换，质量更优且推理显著加速。
摘要：
零样本语音转换（VC）旨在将源说话人的音色迁移给任意未见过的说话人，同时保留原始语言内容。尽管近期基于语言模型或扩散模型的零样本VC方法取得了进展，但仍存在若干挑战：1）当前方法主要关注适配未见过说话人的音色，无法将风格和音色独立迁移给不同的未见过说话人；2）由于自回归建模方法或需要大量采样步骤，这些方法往往推理速度较慢；3）转换样本的质量和相似度仍不能完全令人满意。为应对这些挑战，我们提出了一种名为StableVC的风格可控零样本VC方法，旨在将源语音的音色和风格迁移给不同的未见过目标说话人。具体而言，我们将语音分解为语言内容、音色和风格，然后采用条件流匹配模块，基于这些分解后的特征重建高质量梅尔频谱图。为了以零样本方式有效捕获音色和风格，我们引入了一种带有自适应门控的新型双重注意力机制，而非使用传统的特征拼接。通过这种非自回归设计，StableVC能够高效捕获来自不同未见过说话人的复杂音色和风格，并以显著快于实时的速度生成高质量语音。实验表明，我们提出的StableVC在零样本VC中优于最先进的基线系统，并实现了对来自不同未见过说话人的音色和风格的灵活控制。此外，与自回归基线和基于扩散的基线相比，StableVC的采样速度分别快约25倍和1.65倍。
英文摘要：
Zero-shot voice conversion (VC) aims to transfer the timbre from the source speaker to an arbitrary unseen speaker while preserving the original linguistic content. Despite recent advancements in zero-shot VC using language model-based or diffusion-based approaches, several challenges remain: 1) current approaches primarily focus on adapting timbre from unseen speakers and are unable to transfer style and timbre to different unseen speakers independently; 2) these approaches often suffer from slower inference speeds due to the autoregressive modeling methods or the need for numerous sampling steps; 3) the quality and similarity of the converted samples are still not fully satisfactory. To address these challenges, we propose a style controllable zero-shot VC approach named StableVC, which aims to transfer timbre and style from source speech to different unseen target speakers. Specifically, we decompose speech into linguistic content, timbre, and style, and then employ a conditional flow matching module to reconstruct the high-quality mel-spectrogram based on these decomposed features. To effectively capture timbre and style in a zero-shot manner, we introduce a novel dual attention mechanism with an adaptive gate, rather than using conventional feature concatenation. With this non-autoregressive design, StableVC can efficiently capture the intricate timbre and style from different unseen speakers and generate high-quality speech significantly faster than real-time. Experiments demonstrate that our proposed StableVC outperforms state-of-the-art baseline systems in zero-shot VC and achieves flexible control over timbre and style from different unseen speakers. Moreover, StableVC offers approximately 25x and 1.65x faster sampling compared to autoregressive and diffusion-based baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
