arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2008.02070eess.AScs.LGcs.SD
收藏

通过使用对齐音素的强条件反射的基于内容的歌唱声音源分离

Content based singing voice source separation via strong conditioning using aligned phonemes

STMS UMR9912, Ircam/CNRS/SU(STMS UMR9912,Ircam/CNRS/SU);LTCI, Institut Polytechnique de Paris(巴黎理工学院 LTCI)

Gabriel Meseguer-Brocal, Geoffroy Peeters

摘要:

最近,随着神经网络的引入和包含混合源和分离源的大型多轨数据集的可用性,信息源分离获得了新的兴趣。这些方法使用关于目标源的先验信息来改进分离。从历史上看,音乐信息检索的研究人员主要集中在乐谱知情的源分离,但最近的方法探索歌词知情的源分离。然而,由于缺乏具有时间对齐歌词的多轨道数据集,模型使用非对齐歌词的弱条件反射。在本文中,我们提出了一个多模态的多轨数据集与歌词对齐的时间在单词级的语音信息,以及探索强条件反射使用对齐的音素。我们的模型遵循U-Net架构,并将音乐混合物的幅度谱图和具有对齐语音信息的矩阵作为输入。音素矩阵被嵌入以获得控制逐行线性调制(FilM)层的参数。这些层调节U-Net特征映射,以通过仿射变换使分离过程适应不同音素的存在。我们表明,音素条件反射可以成功地应用于提高歌唱声源分离。

英文摘要:

Informed source separation has recently gained renewed interest with the introduction of neural networks and the availability of large multitrack datasets containing both the mixture and the separated sources. These approaches use prior information about the target source to improve separation. Historically, Music Information Retrieval researchers have focused primarily on score-informed source separation, but more recent approaches explore lyrics-informed source separation. However, because of the lack of multitrack datasets with time-aligned lyrics, models use weak conditioning with non-aligned lyrics. In this paper, we present a multimodal multitrack dataset with lyrics aligned in time at the word level with phonetic information as well as explore strong conditioning using the aligned phonemes. Our model follows a U-Net architecture and takes as input both the magnitude spectrogram of a musical mixture and a matrix with aligned phonetic information. The phoneme matrix is embedded to obtain the parameters that control Feature-wise Linear Modulation (FiLM) layers. These layers condition the U-Net feature maps to adapt the separation process to the presence of different phonemes via affine transformations. We show that phoneme conditioning can be successfully applied to improve singing voice source separation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑