FNSE-SBGAN:基于薛定谔桥和生成对抗网络的远场语音增强
FNSE-SBGAN: Far-field Speech Enhancement with Schrodinger Bridge and Generative Adversarial Networks
AI总结:
针对神经语音增强模型对真实环境混合语音泛化能力有限的问题,本研究提出FNSE-SBGAN框架,整合薛定谔桥扩散模型与GANs,直接在真实混合语音上训练,显著提升远场到近场语音增强性能并建立新基准。
摘要:
当前主流的神经语音增强方法主要采用全监督深度学习,使用模拟的远场带噪混响语音与干净语音对。然而,这些模型对真实环境中录制的混合语音的泛化能力往往有限。为解决此问题,本研究探索直接在真实混合语音上训练增强模型。具体而言,我们重新研究单通道远场到近场语音增强(FNSE)任务,聚焦于具有低信噪比(SNR)、高混响和中高频衰减特征的真实数据。我们提出FNSE-SBGAN框架,整合基于薛定谔桥(SB)的扩散模型与生成对抗网络(GANs)。该框架在多种指标和主观评估中取得最先进性能,与远场信号相比,字符错误率(CER)最多降低14.58%。实验结果表明,FNSE-SBGAN保留了优异的主观质量,为真实环境远场语音增强建立了新基准。此外,我们引入一种利用时频域矩阵秩分析的评估框架,为模型性能提供系统见解,并揭示不同生成方法的优缺点。
英文摘要:
The prevailing method for neural speech enhancement predominantly utilizes fully-supervised deep learning with simulated pairs of far-field noisy-reverberant speech and clean speech. Nonetheless, these models frequently demonstrate restricted generalizability to mixtures recorded in real-world conditions. To address this issue, this study investigates training enhancement models directly on real mixtures. Specifically, we revisit the single-channel far-field to near-field speech enhancement (FNSE) task, focusing on real-world data characterized by low signal-to-noise ratio (SNR), high reverberation, and mid-to-high frequency attenuation. We propose FNSE-SBGAN, a framework that integrates a Schrodinger Bridge (SB)-based diffusion model with generative adversarial networks (GANs). Our approach achieves state-of-the-art performance across various metrics and subjective evaluations, significantly reducing the character error rate (CER) by up to 14.58% compared to far-field signals. Experimental results demonstrate that FNSE-SBGAN preserves superior subjective quality and establishes a new benchmark for real-world far-field speech enhancement. Additionally, we introduce an evaluation framework leveraging matrix rank analysis in the time-frequency domain, providing systematic insights into model performance and revealing the strengths and weaknesses of different generative methods.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。