Bridge-SR: 用于高效语音超分辨率的薛定谔桥方法
Bridge-SR: Schrödinger Bridge for Efficient SR
AI总结:
针对现有语音超分辨率方法需额外压缩网络或合成质量与速度不足的问题,提出基于薛定谔桥的Bridge-SR系统,以低分辨率波形为先验,结合轻量网络与关键优化策略,在VCTK数据集上实现了更优的质量与推理效率。
摘要:
语音超分辨率(SR)是从低分辨率波形生成更高采样率波形的任务,是语音修复领域长期存在的关键任务。已有研究在不同数据空间探索了语音超分辨率方法,但这些方法要么需要额外的压缩网络,要么合成质量和推理速度存在局限。受概率生成模型最新进展的启发,我们提出Bridge-SR,这是一种在语音波形域实现的新型高效任意采样率转48kHz超分辨率系统。我们采用易处理的Schrödinger Bridge(薛定谔桥)模型,将观测到的低分辨率波形作为先验,该先验本身对高分辨率目标具有信息价值。通过优化一个轻量级网络来学习从先验到目标的得分函数,我们通过充分利用低分辨率观测中包含的指导性内容的数据到数据生成过程,实现了高效的波形超分辨率。此外,我们还明确了噪声调度、数据缩放和辅助损失函数的重要性,这些因素进一步提升了基于桥接的系统的超分辨率质量。在基准数据集VCTK上开展的实验验证了我们系统的效率:(1)在样本质量方面,Bridge-SR采用轻量级网络骨干(参数量1.7M),在不同超分辨率设置下的表现均优于多个强基线方法;(2)在推理速度方面,我们的4步合成效果优于8步条件扩散模型对应方法(LSD指标:0.911对比0.927)。演示页面见https://bridge-sr.github.io。
英文摘要:
Speech super-resolution (SR), which generates a waveform at a higher sampling rate from its low-resolution version, is a long-standing critical task in speech restoration. Previous works have explored speech SR in different data spaces, but these methods either require additional compression networks or exhibit limited synthesis quality and inference speed. Motivated by recent advances in probabilistic generative models, we present Bridge-SR, a novel and efficient any-to-48kHz SR system in the speech waveform domain. Using tractable Schrödinger Bridge models, we leverage the observed low-resolution waveform as a prior, which is intrinsically informative for the high-resolution target. By optimizing a lightweight network to learn the score functions from the prior to the target, we achieve efficient waveform SR through a data-to-data generation process that fully exploits the instructive content contained in the low-resolution observation. Furthermore, we identify the importance of the noise schedule, data scaling, and auxiliary loss functions, which further improve the SR quality of bridge-based systems. The experiments conducted on the benchmark dataset VCTK demonstrate the efficiency of our system: (1) in terms of sample quality, Bridge-SR outperforms several strong baseline methods under different SR settings, using a lightweight network backbone (1.7M); (2) in terms of inference speed, our 4-step synthesis achieves better performance than the 8-step conditional diffusion counterpart (LSD: 0.911 vs 0.927). Demo at https://bridge-sr.github.io.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。