用扩散模型求解音频逆问题
Solving Audio Inverse Problems with a Diffusion Model
摘要:
本文介绍了CQT-Diff,一种数据驱动的生成音频模型,一旦经过训练,可以用于在问题不可知的环境中解决各种不同的音频逆问题。CQT-Diff是一个神经扩散模型,其架构经过精心构建,以利用音乐中的音高等变对称性。这是通过用可逆的恒定Q变换(CQT)预处理模型来实现的,CQT的频率间隔轴将音高等方差表示为平移等方差。所提出的方法进行评估,在三个不同的和不同的任务:音频带宽扩展,修复和declipping的客观和主观的指标。结果表明,CQT-Diff在音频带宽扩展方面优于比较的基线和消融,并且在没有重新训练的情况下,在音频修复和去唇方面提供了与现代基线相比具有竞争力的性能。这项工作代表了第一个基于扩散的一般框架,用于解决音频处理中的逆问题。
英文摘要:
This paper presents CQT-Diff, a data-driven generative audio model that can, once trained, be used for solving various different audio inverse problems in a problem-agnostic setting. CQT-Diff is a neural diffusion model with an architecture that is carefully constructed to exploit pitch-equivariant symmetries in music. This is achieved by preconditioning the model with an invertible Constant-Q Transform (CQT), whose logarithmically-spaced frequency axis represents pitch equivariance as translation equivariance. The proposed method is evaluated with objective and subjective metrics in three different and varied tasks: audio bandwidth extension, inpainting, and declipping. The results show that CQT-Diff outperforms the compared baselines and ablations in audio bandwidth extension and, without retraining, delivers competitive performance against modern baselines in audio inpainting and declipping. This work represents the first diffusion-based general framework for solving inverse problems in audio processing.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。