arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.18747cs.SDcs.AIcs.IRcs.LGeess.AS
收藏

一种用于超越四个音轨的音乐源分离的与音轨无关的单解码器系统

A Stem-Agnostic Single-Decoder System for Music Source Separation Beyond Four Stems

Georgia Institute of Technology(佐治亚理工学院)

Karn N. Watcharasupat, Alexander Lerch

AI总结:

提出 Banquet 系统,通过单解码器和基于查询的设置结合 PaSST 模型,实现超越传统四音轨的灵活音乐源分离,在更少参数下性能接近或超越复杂基线,并支持长尾乐器提取。

摘要:

尽管近期在音频源分离的多个子任务上取得了显著进展,但很少有音乐源分离系统能支持超越四个音轨——人声、鼓、贝斯和其他(VDBO)设置的分离。在目前极少数支持超越此设置的源分离系统中,大多数仍继续依赖一种不灵活的解码器设置,该设置只能支持固定的预定义音轨集合。在这些不灵活的系统中增加音轨支持相应地需要增加计算复杂度,使得这些系统在计算上无法扩展至长尾乐器。在本研究中,我们提出了 Banquet,这是一个仅使用一个解码器即可实现多个音轨源分离的系统。我们将一个频带分割源分离模型扩展为在基于查询的设置下工作,并与音乐乐器识别 PaSST 模型协同配合。在 MoisesDB 数据集上,Banquet 仅具有 24.9M 可训练参数,在 VDBO 音轨上接近了复杂度显著更高的 6 音轨 Hybrid Transformer Demucs 的性能水平,并在吉他和钢琴上超越了它。基于查询的设置允许对诸如原声吉他等狭窄乐器类别进行分离,并能成功应用于簧片和风琴等较不常见音轨的提取。实现代码可在 https://github.com/kwatcharasupat/query-bandit 获取。

英文摘要:

Despite significant recent progress across multiple subtasks of audio source separation, few music source separation systems support separation beyond the four-stem vocals, drums, bass, and other (VDBO) setup. Of the very few current systems that support source separation beyond this setup, most continue to rely on an inflexible decoder setup that can only support a fixed pre-defined set of stems. Increasing stem support in these inflexible systems correspondingly requires increasing computational complexity, rendering extensions of these systems computationally infeasible for long-tail instruments. In this work, we propose Banquet, a system that allows source separation of multiple stems using just one decoder. A bandsplit source separation model is extended to work in a query-based setup in tandem with a music instrument recognition PaSST model. On the MoisesDB dataset, Banquet, at only 24.9 M trainable parameters, approached the performance level of the significantly more complex 6-stem Hybrid Transformer Demucs on VDBO stems and outperformed it on guitar and piano. The query-based setup allows for the separation of narrow instrument classes such as clean acoustic guitars, and can be successfully applied to the extraction of less common stems such as reeds and organs. Implementation is available at https://github.com/kwatcharasupat/query-bandit.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑