SynthVC:利用合成数据实现端到端低延迟流式语音转换
SynthVC: Leveraging Synthetic Data for End-to-End Low Latency Streaming Voice Conversion
AI总结:
提出流式端到端语音转换框架SynthVC,利用预训练零样本VC模型生成的合成并行数据直接学习音色转换,无需显式内容-说话人分离或识别模块,在自然度与说话人相似度上优于基线,端到端延迟仅77.1 ms。
摘要:
语音转换(VC)旨在改变说话人音色,同时保留语言内容。尽管近期的 VC 模型表现优异,但大多数在实时流式场景中仍面临困难,原因在于高延迟、依赖 ASR 模块或复杂的说话人解耦,这通常会导致音色泄漏或自然度下降。我们提出了 SynthVC,这是一个流式端到端 VC 框架,直接从预训练零样本 VC 模型生成的合成并行数据中学习说话人音色转换。该设计消除了对显式内容-说话人分离或识别模块的需求。SynthVC 构建于神经音频编解码器架构之上,支持低延迟流式推理且输出保真度高。实验结果表明,SynthVC 在自然度和说话人相似度上均优于基线流式 VC 系统,实现了仅 77.1 ms 的端到端延迟。
英文摘要:
Voice Conversion (VC) aims to modify a speaker's timbre while preserving linguistic content. While recent VC models achieve strong performance, most struggle in real-time streaming scenarios due to high latency, dependence on ASR modules, or complex speaker disentanglement, which often results in timbre leakage or degraded naturalness. We present SynthVC, a streaming end-to-end VC framework that directly learns speaker timbre transformation from synthetic parallel data generated by a pre-trained zero-shot VC model. This design eliminates the need for explicit content-speaker separation or recognition modules. Built upon a neural audio codec architecture, SynthVC supports low-latency streaming inference with high output fidelity. Experimental results show that SynthVC outperforms baseline streaming VC systems in both naturalness and speaker similarity, achieving an end-to-end latency of just 77.1 ms.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。