arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.07077eess.AScs.SD
收藏

来自音乐音频的多源对比学习

Multi-Source Contrastive Learning from Musical Audio

Institute of Language and Speech Proc., Athena Research Center(雅典娜研究中心语言与语音研究所);Institute of Robotics, Athena Research Center(雅典娜研究中心机器人研究所);School of ECE, National Technical University of Athens(雅典国家技术大学电气与计算机工程学院)

Christos Garoufis, Athanasia Zlatintsi, Petros Maragos

AI总结:

本文提出一种基于音乐源关联的多源对比学习预训练方法,通过修改COLA框架和对比损失,在三个下游任务中取得竞争性结果并加速收敛。

摘要:

对比学习构成了自监督学习的一个新兴分支,它通过利用大量未标记数据,学习一个潜在空间,在该空间中同一样本的不同视图对相互关联。在本文中,我们提出将音乐源关联作为对比音乐表示学习背景下的一种配对生成策略。为此,我们修改了COLA(一种广泛使用的对比学习音频框架),以学习将歌曲片段与随机选择并自动提取的人声或乐器源进行关联。我们进一步引入了一种对对比损失的新颖修改,以纳入关于特定源存在或缺失的信息。我们在三个不同的下游任务(音乐自动标注、乐器分类和音乐流派分类)中使用公开可用的Magna-Tag-A-Tune(MTAT)作为源数据集进行的实验评估,取得了与现有文献方法相比具有竞争力的结果,同时网络收敛速度更快。结果还表明,这种预训练方法可以根据所选音乐源被引导至特定特征,同时也依赖于分离源的质量。

英文摘要:

Contrastive learning constitutes an emerging branch of self-supervised learning that leverages large amounts of unlabeled data, by learning a latent space, where pairs of different views of the same sample are associated. In this paper, we propose musical source association as a pair generation strategy in the context of contrastive music representation learning. To this end, we modify COLA, a widely used contrastive learning audio framework, to learn to associate a song excerpt with a stochastically selected and automatically extracted vocal or instrumental source. We further introduce a novel modification to the contrastive loss to incorporate information about the existence or absence of specific sources. Our experimental evaluation in three different downstream tasks (music auto-tagging, instrument classification and music genre classification) using the publicly available Magna-Tag-A-Tune (MTAT) as a source dataset yields competitive results to existing literature methods, as well as faster network convergence. The results also show that this pre-training method can be steered towards specific features, according to the selected musical source, while also being dependent on the quality of the separated sources.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑