arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.08977eess.AScs.SD
收藏

通用语音内容分解

Universal Speech Content Factorization

Center for Language and Speech Processing, Johns Hopkins University, USA(约翰霍普金斯大学语言与语音处理中心);Human Language Technology Center of Excellence (COE), Johns Hopkins University, USA(约翰霍普金斯大学人类语言技术卓越中心(COE))

Henry Li Xinyuan, Zexin Cai, Lin Zhang, Leibny Paola García-Perera, Berrak Sisman, Sanjeev Khudanpur, Nicholas Andrews, Matthew Wiesner

AI总结:

本文提出USCF方法,通过线性可逆方法提取低秩语音表示,抑制说话人音色同时保留语音内容。该方法扩展了语音内容分解,通过最小二乘优化学习通用语音到内容映射,并从少量目标语音中推导出说话人特定转换。

摘要:

我们提出通用语音内容分解(USCF),一种简单且可逆的线性方法,用于提取低秩语音表示,在其中抑制说话人音色同时保留语音内容。USCF通过最小二乘优化学习通用语音到内容映射,扩展了语音内容分解,一种封闭集语音转换(VC)方法,到开放集设置。我们通过嵌入分析显示USCF有效去除说话人依赖性变化。作为零样本语音转换系统,USCF在可懂度、自然度和说话人相似性方面与需要大量目标说话人数据或额外神经网络训练的方法相媲美。最后,我们证明作为训练高效的音色分离语音特征,USCF特征可作为训练音色提示文本到语音模型的声学表示。语音样本和代码已公开提供。

英文摘要:

We propose Universal Speech Content Factorization (USCF), a simple and invertible linear method for extracting a low-rank speech representation in which speaker timbre is suppressed while phonetic content is preserved. USCF extends Speech Content Factorization, a closed-set voice conversion (VC) method, to an open-set setting by learning a universal speech-to-content mapping via least-squares optimization and deriving speaker-specific transformations from only a few seconds of target speech. We show through embedding analysis that USCF effectively removes speaker-dependent variation. As a zero-shot VC system, USCF achieves competitive intelligibility, naturalness, and speaker similarity compared to methods that require substantially more target-speaker data or additional neural training. Finally, we demonstrate that as a training-efficient timbre-disentangled speech feature, USCF features can serve as the acoustic representation for training timbre-prompted text-to-speech models. Speech samples and code are publicly available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑