从真实歌手识别到克隆歌手识别
From Real to Cloned Singer Identification
AI总结:
针对克隆歌手声音带来的人格权威胁,本文提出三种采用歌手级对比学习训练的嵌入模型用于识别合成声音中的原唱,发现模型真实歌手识别能力强但对克隆声音识别性能下降,凸显研究歌手识别系统偏差的必要性。
摘要:
近年来,流行歌手的克隆声音听起来越来越逼真,且已逐渐普及。但由于人格权相关问题,这类克隆声音给行业带来了威胁。因此,亟需能够识别合成声音中原唱歌手的方法。本文研究了歌手识别方法可如何应用于该任务。我们提出了三种嵌入模型,均采用歌手级对比学习方案训练,其中正样本对由同一位歌手的人声片段构成。第一种模型的输入片段可以是混音,第二种为人声干声,第三种则同时使用两者。实验证明,三种模型都具备极强的真实歌手识别能力。但在对我们评估集中的歌手克隆版本进行分类时,模型性能会出现下降,其中以混音作为输入的模型性能下降尤为明显。这些发现凸显了我们有必要理解歌手识别系统中存在的偏差,以及这类偏差会如何影响音乐领域中声音深度伪造内容的识别。
英文摘要:
Cloned voices of popular singers sound increasingly realistic and have gained popularity over the past few years. They however pose a threat to the industry due to personality rights concerns. As such, methods to identify the original singer in synthetic voices are needed. In this paper, we investigate how singer identification methods could be used for such a task. We present three embedding models that are trained using a singer-level contrastive learning scheme, where positive pairs consist of segments with vocals from the same singers. These segments can be mixtures for the first model, vocals for the second, and both for the third. We demonstrate that all three models are highly capable of identifying real singers. However, their performance deteriorates when classifying cloned versions of singers in our evaluation set. This is especially true for models that use mixtures as an input. These findings highlight the need to understand the biases that exist within singer identification systems, and how they can influence the identification of voice deepfakes in music.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。