Thunder:使用布朗桥的单步逆向统一回归-扩散语音增强
Thunder : Unified Regression-Diffusion Speech Enhancement with a Single Reverse Step using Brownian Bridge
AI总结:
本文提出统一回归-扩散模型Thunder,利用布朗桥过程实现单步逆向语音增强,通过预测干净语音替代分数函数解决梯度不稳定问题,以更小模型和更少步骤达到竞争性能。
摘要:
基于扩散的语音增强已展现出有前景的结果,但可能面临推理时间较慢的问题。使用基于回归模型生成的增强音频来初始化扩散过程,可用于减少所需的计算步骤。然而,这些方法通常需要一个回归模型,进一步增加了系统的复杂性。我们提出Thunder,一个统一的回归-扩散模型,利用布朗桥过程使模型能够在两种模式下运行。通过将扩散时间步设置为接近1,可以进入回归模式。然而,标准的基于分数的扩散建模由于梯度不稳定性,在此设置下表现不佳。为缓解这一问题,我们修改扩散模型以预测干净语音而非分数函数,从而以更紧凑的模型规模和更少的逆向步骤实现具有竞争力的性能。
英文摘要:
Diffusion-based speech enhancement has shown promising results, but can suffer from a slower inference time. Initializing the diffusion process with the enhanced audio generated by a regression-based model can be used to reduce the computational steps required. However, these approaches often necessitate a regression model, further increasing the system's complexity. We propose Thunder, a unified regression-diffusion model that utilizes the Brownian bridge process which can allow the model to act in both modes. The regression mode can be accessed by setting the diffusion time step closed to 1. However, the standard score-based diffusion modeling does not perform well in this setup due to gradient instability. To mitigate this problem, we modify the diffusion model to predict the clean speech instead of the score function, achieving competitive performance with a more compact model size and fewer reverse steps.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。