利用多样化基于语义的音频预训练模型实现歌声转换
Leveraging Diverse Semantic-based Audio Pretrained Models for Singing Voice Conversion
AI总结：
针对歌声转换中语义预训练特征适配性不足的问题，研究发现不同语义音频预训练模型的知识具有多样性与互补性，提出基于多样化语义特征融合的DSFF-SVC框架，可有效提升多种现有SVC模型在真实场景下的转换效果。
摘要：
歌声转换（Singing Voice Conversion, SVC）是一种能让任意演唱者演唱任意歌曲的技术。要实现这一目标，从源音频中获取与说话人无关的表征至关重要，这也是一项重大挑战。常见的解决方案是使用基于语义的音频预训练模型作为特征提取器，但提取的特征能在多大程度上满足SVC的需求仍是一个悬而未决的问题，包括它们准确建模旋律和歌词的能力、底层声学信息的说话人无关性，以及在真实声学环境下的鲁棒性。本研究详细探究了经典基于语义的预训练模型中的知识，发现不同模型的知识具有多样性，且对SVC具有互补性。基于此，我们设计了一个基于多样化语义特征融合的歌声转换框架（DSFF-SVC）。实验结果表明，DSFF-SVC具有通用性，能够改进多种现有SVC模型，尤其是在具有挑战性的真实转换任务中。我们的演示网站位于https://diversesemanticsvc.github.io/。
英文摘要：
Singing Voice Conversion (SVC) is a technique that enables any singer to perform any song. To achieve this, it is essential to obtain speaker-agnostic representations from the source audio, which poses a significant challenge. A common solution involves utilizing a semantic-based audio pretrained model as a feature extractor. However, the degree to which the extracted features can meet the SVC requirements remains an open question. This includes their capability to accurately model melody and lyrics, the speaker-independency of their underlying acoustic information, and their robustness for in-the-wild acoustic environments. In this study, we investigate the knowledge within classical semantic-based pretrained models in much detail. We discover that the knowledge of different models is diverse and can be complementary for SVC. Based on the above, we design a Singing Voice Conversion framework based on Diverse Semantic-based Feature Fusion (DSFF-SVC). Experimental results demonstrate that DSFF-SVC can be generalized and improve various existing SVC models, particularly in challenging real-world conversion tasks. Our demo website is available at https://diversesemanticsvc.github.io/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
