本文提出频域稀疏压缩网络 SCNet,通过子带划分与差异化压缩建模提升超宽带音乐源分离性能,在 MUSDB18-HQ 上以更低 CPU 推理耗时取得 9.0 dB SDR。
摘要:
基于深度学习的方法已在音乐源分离中取得显著成果。然而,在超宽带音乐源分离中,如何在保持较低模型复杂度的同时获得良好结果仍然具有挑战性。以往工作要么忽视了不同子带之间的差异,要么未能充分解决生成子带特征时的信息丢失问题。本文提出 SCNet,这是一种新颖的频域网络,可显式地将混合信号的频谱图划分为若干子带,并引入基于稀疏性的编码器来对不同频带进行建模。我们对信息较少的子带使用更高的压缩比,以提高信息密度,并集中对信息较多的子带进行建模。通过这种方式,可以在更低计算消耗下显著提升分离性能。实验结果表明,所提出模型在不使用额外数据的情况下,在 MUSDB18-HQ 数据集上达到了 9.0 dB 的信号失真比(SDR),优于现有最先进方法。具体而言,SCNet 的 CPU 推理时间仅为 HT Demucs(此前最先进模型之一)的 48%。
英文摘要:
Deep learning-based methods have made significant achievements in music source separation. However, obtaining good results while maintaining a low model complexity remains challenging in super wide-band music source separation. Previous works either overlook the differences in subbands or inadequately address the problem of information loss when generating subband features. In this paper, we propose SCNet, a novel frequency-domain network to explicitly split the spectrogram of the mixture into several subbands and introduce a sparsity-based encoder to model different frequency bands. We use a higher compression ratio on subbands with less information to improve the information density and focus on modeling subbands with more information. In this way, the separation performance can be significantly improved using lower computational consumption. Experiment results show that the proposed model achieves a signal to distortion ratio (SDR) of 9.0 dB on the MUSDB18-HQ dataset without using extra data, which outperforms state-of-the-art methods. Specifically, SCNet's CPU inference time is only 48% of HT Demucs, one of the previous state-of-the-art models.
Shenzhen International Graduate School, Tsinghua University, Shenzhen, China(清华大学深圳国际研究生院);Tencent AI Lab(腾讯AI实验室);The Chinese University of Hong Kong, Hong Kong SAR, China(香港中文大学)
MusicGen-Stem: Multi-stem music generation and edition through autoregressive modeling
Meta;Sorbonne Univ.(索邦大学);UMR STMS IRCAM-CNRS;Univ. de Lorraine(洛林大学);CNRS(法国国家科学研究中心);Inria(法国国家信息与自动化研究所);Loria(洛里亚研究所);Hebrew Univ. of Jerusalem(耶路撒冷希伯来大学)
DTT-BSR: GAN-based DTTNet with RoPE Transformer Enhancement for Music Source Restoration
School of Electronic Information, Wuhan University, China(武汉大学电子信息学院);Southwestern University of Finance and Economics, China(西南财经大学);Tampere University, Finland(塔尔基大学);MiLM Plus, Xiaomi Inc., China(小米公司MiLM Plus)