arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.05064cs.SDcs.LGeess.AS
收藏

使用自监督技术的歌手身份表示学习

Singer Identity Representation Learning using Self-Supervised Techniques

Telecom Paris(巴黎电信学院);Institut Polytechnique de Paris(巴黎综合理工学院);Sony Computer Science Laboratories Paris(索尼巴黎计算机科学实验室)

Bernardo Torres, Stefan Lattner, Gaël Richard

AI总结:

本文提出一个自监督框架训练歌手身份编码器,通过数据增强提取对音高和内容变化不变的表示,在歌手相似性与识别任务上优于基线模型并促进歌声研究。

摘要:

在使用语音数据创建声音身份表示方面已取得重大进展。然而,对于歌声,尚未达到同等水平的进展。为了弥合这一差距,我们提出了一个框架,用于训练歌手身份编码器,以提取适用于各种与歌唱相关任务的表示,例如歌声相似性和合成。我们在大量孤立的声乐音轨上探索了不同的自监督学习技术,并在训练期间应用数据增强,以确保表示对音高和内容变化具有不变性。我们在多个数据集上评估了所得表示在歌手相似性和识别任务中的质量,特别强调了域外泛化能力。我们提出的框架产生了高质量的嵌入,在44.1 kHz下运行时,在歌声方面优于说话人验证和wav2vec 2.0预训练基线。我们发布了代码和训练好的模型,以促进对歌声及相关领域的进一步研究。

英文摘要:

Significant strides have been made in creating voice identity representations using speech data. However, the same level of progress has not been achieved for singing voices. To bridge this gap, we suggest a framework for training singer identity encoders to extract representations suitable for various singing-related tasks, such as singing voice similarity and synthesis. We explore different self-supervised learning techniques on a large collection of isolated vocal tracks and apply data augmentations during training to ensure that the representations are invariant to pitch and content variations. We evaluate the quality of the resulting representations on singer similarity and identification tasks across multiple datasets, with a particular emphasis on out-of-domain generalization. Our proposed framework produces high-quality embeddings that outperform both speaker verification and wav2vec 2.0 pre-trained baselines on singing voice while operating at 44.1 kHz. We release our code and trained models to facilitate further research on singing voice and related areas.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑