SEF-VC：基于交叉注意力的无需说话人嵌入的零样本语音转换
SEF-VC: Speaker Embedding Free Zero-Shot Voice Conversion with Cross Attention
AI总结：
针对零样本语音转换中说话人相似度不足的问题，提出无需说话人嵌入的SEF-VC模型，通过位置无关交叉注意力融合参考音色并非自回归重建波形，在短参考下也能实现更优的语音质量与目标相似度。
摘要：
零样本语音转换（VC）旨在将源说话人的音色转换为任意未见过的目标说话人的音色，同时保持语言内容不变。尽管可以通过提供目标说话人的嵌入来控制生成语音的音色，但说话人相似度仍不及真实录制音频。本文提出SEF-VC，一种无需说话人嵌入的语音转换模型，该模型通过强大的位置无关交叉注意力机制从参考语音中学习并融合说话人音色，随后以非自回归方式从HuBERT语义令牌重建波形。SEF-VC的简洁设计提升了训练稳定性和语音转换性能。客观与主观评估表明，即便参考语音极短，SEF-VC生成的语音质量高，与目标参考的相似度优于强零样本VC基线。
英文摘要：
Zero-shot voice conversion (VC) aims to transfer the source speaker timbre to arbitrary unseen target speaker timbre, while keeping the linguistic content unchanged. Although the voice of generated speech can be controlled by providing the speaker embedding of the target speaker, the speaker similarity still lags behind the ground truth recordings. In this paper, we propose SEF-VC, a speaker embedding free voice conversion model, which is designed to learn and incorporate speaker timbre from reference speech via a powerful position-agnostic cross-attention mechanism, and then reconstruct waveform from HuBERT semantic tokens in a non-autoregressive manner. The concise design of SEF-VC enhances its training stability and voice conversion performance. Objective and subjective evaluations demonstrate the superiority of SEF-VC to generate high-quality speech with better similarity to target reference than strong zero-shot VC baselines, even for very short reference speeches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
