FlowSE：基于流匹配的高效高质量语音增强
FlowSE: Efficient and High-Quality Speech Enhancement via Flow Matching
AI总结：
FlowSE提出基于流匹配的语音增强模型，单次传递学习噪声到干净语音的连续变换，降低推理延迟并保持高质量，显著优于现有生成方法。
摘要：
生成模型在音频任务中表现出色，采用了语言模型、扩散和流匹配等方法。然而，现有的语音增强（SE）生成方法面临显著挑战：基于语言模型的方法遭受量化损失，导致说话人相似度和可懂度受损，而扩散模型需要复杂的训练和高推理延迟。为解决这些问题，我们提出了FlowSE，一种基于流匹配的语音增强模型。流匹配在单次传递中学习噪声和干净语音分布之间的连续变换，显著降低推理延迟，同时保持高质量重建。具体而言，FlowSE在带噪梅尔频谱图和可选字符序列上训练，优化条件流匹配损失，以真实梅尔频谱图作为监督。它隐式学习语音的时频结构和文本-语音对齐。在推理时，FlowSE可以在有或无文本信息的情况下运行，在两种场景下均取得令人印象深刻的结果，当有转录可用时进一步改进。大量实验表明，FlowSE显著优于最先进的生成方法，为基于生成的语音增强建立了新范式，并展示了流匹配推动该领域发展的潜力。我们的代码、预训练检查点和音频样本均可获得。
英文摘要：
Generative models have excelled in audio tasks using approaches such as language models, diffusion, and flow matching. However, existing generative approaches for speech enhancement (SE) face notable challenges: language model-based methods suffer from quantization loss, leading to compromised speaker similarity and intelligibility, while diffusion models require complex training and high inference latency. To address these challenges, we propose FlowSE, a flow-matching-based model for SE. Flow matching learns a continuous transformation between noisy and clean speech distributions in a single pass, significantly reducing inference latency while maintaining high-quality reconstruction. Specifically, FlowSE trains on noisy mel spectrograms and optional character sequences, optimizing a conditional flow matching loss with ground-truth mel spectrograms as supervision. It implicitly learns speech's temporal-spectral structure and text-speech alignment. During inference, FlowSE can operate with or without textual information, achieving impressive results in both scenarios, with further improvements when transcripts are available. Extensive experiments demonstrate that FlowSE significantly outperforms state-of-the-art generative methods, establishing a new paradigm for generative-based SE and demonstrating the potential of flow matching to advance the field. Our code, pre-trained checkpoints, and audio samples are available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
