arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2510.03025eess.AScs.SD
收藏

CVSM:对比人声相似性建模

CVSM: Contrastive Vocal Similarity Modeling

School of Electrical and Computer Engineering, National Technical University of Athens(电气与计算机工程学院,国家技术大学雅典分校);Robotics Institute, Athena Research Center(机器人研究所,阿提卡研究中心);HERON - Hellenic Robotics Center of Excellence(HERON—希腊机器人 excellence 中心)

Christos Garoufis, Athanasia Zlatintsi, Petros Maragos

AI总结:

提出对比自监督框架 CVSM,通过人声与含相同人声的音乐混音对齐来学习音乐表征,并在下游任务和用户研究中验证其人声相似性建模效果。

摘要:

跨各个领域的大型无标注数据集的可用性,促进了大量方法的发展,这些方法通过自监督预训练来学习适用于多个目标(下游)任务的表征。在本工作中,我们提出了 CVSM(Contrastive Vocal Similarity Modeling,对比人声相似性建模),这是一种用于音频领域音乐信号表征学习的对比自监督流程,可用于音乐和人声相似性建模。我们的方法在对比框架下运行,最大化人声片段与包含相同人声的音乐混音之间的相似性;我们设计了两种方案:一种是利用艺术家身份信息来采样对比对的标签知情协议,另一种是标签无关方案,该方案通过从随机采样的人声和伴奏片段创建人工混音,并将其与来自相同音频片段的人声配对。我们在衡量人声相似性方面评估了所提出的方法:客观上,通过在一组合适的下游任务上进行线性探测;主观上,通过开展一项用户研究,该研究由查询式推荐场景下不同模型之间的成对比较组成。我们的结果表明,通过 CVSM 学习到的表征在音乐和人声相似性建模方面是有效的,在孤立人声和完整音乐混音上均优于众多基线。此外,尽管预训练期间艺术家身份标签的可用性会在所评估的下游任务和用户研究中带来总体上更一致的性能,但结合真实混音与人工混音进行混合预训练的标签无关 CVSM 变体,在艺术家识别和感知人声相似性方面取得了与标签知情版本相当的性能。

英文摘要:

The availability of large, unlabeled datasets across various domains has contributed to the development of a plethora of methods that learn representations for multiple target (downstream) tasks through self-supervised pre-training. In this work, we introduce CVSM (Contrastive Vocal Similarity Modeling), a contrastive self-supervised procedure for music signal representation learning in the audio domain that can be utilized for musical and vocal similarity modeling. Our method operates under a contrastive framework, maximizing the similarity between vocal excerpts and musical mixtures containing the same vocals; we devise both a label-informed protocol, leveraging artist identity information to sample the contrastive pairs, and a label-agnostic scheme, involving artificial mixture creation from randomly sampled vocal and accompaniment excerpts, which are paired with vocals from the same audio segment. We evaluate our proposed method in measuring vocal similarity both objectively, through linear probing on a suite of appropriate downstream tasks, and subjectively, via conducting a user study consisting of pairwise comparisons between different models in a recommendation-by-query setting. Our results indicate that the representations learned through CVSM are effective in musical and vocal similarity modeling, outperforming numerous baselines across both isolated vocals and complete musical mixtures. Moreover, while the availability of artist identity labels during pre-training leads to overall more consistent performance both in the evaluated downstream tasks and the user study, a label-agnostic CVSM variant incorporating hybrid pre-training with real and artificial mixtures achieves comparable performance to the label-informed one in artist identification and perceived vocal similarity.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑