arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2305.07855eess.AScs.SD
收藏

整体大于部分之和:通过桥接网络改进音乐源分离

The Whole Is Greater than the Sum of Its Parts: Improving Music Source Separation by Bridging Network

Sony AI(索尼AI);Sony Europe B.V.(索尼欧洲公司);Sony Group Corporation(索尼集团公司)

Ryosuke Sawata, Naoya Takahashi, Stefan Uhlich, Shusuke Takahashi, Yuki Mitsufuji

AI总结:

提出由多域损失、桥接操作和组合损失构成的X-scheme,在几乎不增加推理成本和参数的前提下提升多种DNN音乐源分离模型性能,并验证其跨模型与大规模数据泛化能力。

摘要:

本文提出了交叉方案(X-scheme),用于在几乎不增加计算成本的情况下提升基于深度神经网络(DNN)的音乐源分离(MSS)性能。该方案由三个部分组成:(i)多域损失(MDL);(ii)桥接操作,用于耦合各个乐器网络;以及(iii)组合损失(CL)。MDL能够利用音频信号的频域和时域表示。我们通过为每个输出乐器添加桥接路径以共享其信息,对目标网络(即原始基于DNN的MSS的网络架构)进行修改。随后,MDL不仅应用于每个独立声源,也应用于输出声源的组合;因此我们将其称为CL。MDL和CL可以很容易地应用于许多基于DNN的分离方法,因为它们只是训练期间使用的损失函数,不会影响推理阶段。桥接操作不会增加网络中可学习参数的数量。实验结果通过将扩展了X-scheme的Open-Unmix(UMX)、密集连接扩张DenseNet(D3Net)和卷积时域音频分离网络(Conv-TasNet)(分别称为X-UMX、X-D3Net和X-Conv-TasNet)与其原始版本进行比较,证明了该方案的有效性。我们还在大规模数据条件下验证了X-scheme的效果,展示了其相对于数据规模的通用性。在大规模内部数据上训练并用于实验的X-UMX Large(X-UMXL)现已在https://github.com/asteroid-team/asteroid/tree/master/egs/musdb18/X-UMX公开提供。

英文摘要:

This paper presents the crossing scheme (X-scheme) for improving the performance of deep neural network (DNN)-based music source separation (MSS) with almost no increasing calculation cost. It consists of three components: (i) multi-domain loss (MDL), (ii) bridging operation, which couples the individual instrument networks, and (iii) combination loss (CL). MDL enables the taking advantage of the frequency- and time-domain representations of audio signals. We modify the target network, i.e., the network architecture of the original DNN-based MSS, by adding bridging paths for each output instrument to share their information. MDL is then applied to the combinations of the output sources as well as each independent source; hence, we called it CL. MDL and CL can easily be applied to many DNN-based separation methods as they are merely loss functions that are only used during training and do not affect the inference step. Bridging operation does not increase the number of learnable parameters in the network. Experimental results showed that the validity of Open-Unmix (UMX), densely connected dilated DenseNet (D3Net) and convolutional time-domain audio separation network (Conv-TasNet) extended with our X-scheme, respectively called X-UMX, X-D3Net and X-Conv-TasNet, by comparing them with their original versions. We also verified the effectiveness of X-scheme in a large-scale data regime, showing its generality with respect to data size. X-UMX Large (X-UMXL), which was trained on large-scale internal data and used in our experiments, is newly available at https://github.com/asteroid-team/asteroid/tree/master/egs/musdb18/X-UMX.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑