arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2607.00363cs.SDcs.AI

增强流匹配:一种统一引导框架实现高效鲁棒的语音合成

Enhancing Flow Matching with A Unified Guidance Framework for Efficient and Robust Speech Synthesis

Zuoyebang, China(作业帮,中国)

Zuda Yu, Qianhui Xu, Ting Chen, Junhui Zhang, Tao Fu, Hongjiang Yu, Qiangqing Wang, Yang Song

AI总结:

针对流匹配语音合成中的高推理延迟和音色泄露问题,提出统一引导框架,通过数据引导(异构增强)和模型引导(轨迹整流与内在引导目标)提升效率与鲁棒性,实现近三倍加速并改善说话人相似度。

摘要:

流匹配(FM)已成为语音生成的一种强大范式,但仍受限于高推理延迟和音色泄露。为解决这些瓶颈,我们提出了一种统一引导框架,通过两种互补策略增强生成效率和鲁棒性。在数据方面,我们通过异构增强引入数据引导,鼓励模型将语言内容与声学残留分离。同时,我们提出了一种增强的模型引导机制,将轨迹整流与新颖的内在引导目标协同起来。该方法将条件知识蒸馏到网络权重中,并拉直推理轨迹路径,从而消除了无分类器引导(CFG)的开销。实验表明,与最先进的基线相比,我们的框架将推理速度提高了近三倍,同时有效改善了说话人相似度。

英文摘要:

Flow Matching (FM) has emerged as a powerful paradigm for speech generation but remains constrained by high inference latency and timbre leakage. To address these bottlenecks, we propose a unified guidance framework that enhances generation efficiency and robustness through two complementary strategies. On the data front, we introduce Data-guidance via heterogeneous augmentation, encouraging the model to disentangle linguistic content from acoustic residue. In parallel, we propose an enhanced Model-guidance mechanism that synergizes trajectory rectification with a novel intrinsic guidance objective. This approach distills conditional knowledge into network weights and straightens inference trajectory path, thereby eliminating Classifier-Free Guidance (CFG) overhead. Experiments demonstrate that our framework accelerates inference by nearly three times while effectively improving speaker similarity compared to state-of-the-art baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑