基于自监督学习与特征提取的语音与歌唱中嗓音和口音转换统一模型
A Unified Model For Voice and Accent Conversion In Speech and Singing using Self-Supervised Learning and Feature Extraction
AI总结：
本研究提出一种基于HuBERT编码器与HiFi-GAN解码器的统一嗓音转换模型，融合基频与歌手嵌入，可实现语音与歌唱的嗓音及口音转换，提升转换自然度与灵活性，适用于配音、内容创作等多场景。
摘要：
本文提出了一种新型嗓音转换模型，能够同时转换说话和歌唱嗓音。该模型解决了当前系统中的关键挑战，如情感传递、发音与口音变化处理，以及非语言声音的还原。该模型的突出特性之一是能够对同时包含语音和歌唱的混合嗓音样本执行口音转换，可在保留原始内容和韵律的同时改变说话者的口音。所提出的模型采用编码器-解码器架构：编码器基于HuBERT（自监督语音表示模型）处理语音的声学和语言内容，HiFi-GAN（高保真生成对抗网络）解码器则使音频匹配目标说话者的嗓音。模型融合了基频（f0）特征与歌手嵌入向量，以提升性能，同时确保转换过程中保留音高与音调准确性以及嗓音身份。该方法提升了嗓音风格转换的自然度与灵活性，在嗓音配音、内容创作以及文本转语音（TTS）、交互式语音应答（IVR）系统等技术中展现出巨大应用潜力。
英文摘要：
This paper presents a new voice conversion model capable of transforming both speaking and singing voices. It addresses key challenges in current systems, such as conveying emotions, managing pronunciation and accent changes, and reproducing non-verbal sounds. One of the model's standout features is its ability to perform accent conversion on hybrid voice samples that encompass both speech and singing, allowing it to change the speaker's accent while preserving the original content and prosody. The proposed model uses an encoder-decoder architecture: the encoder is based on HuBERT to process the speech's acoustic and linguistic content, while the HiFi-GAN decoder audio matches the target speaker's voice. The model incorporates fundamental frequency (f0) features and singer embeddings to enhance performance while ensuring the pitch & tone accuracy and vocal identity are preserved during transformation. This approach improves how naturally and flexibly voice style can be transformed, showing strong potential for applications in voice dubbing, content creation, and technologies like Text-to-Speech (TTS) and Interactive Voice Response (IVR) systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
