arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.00527eess.AS
收藏

想听你的声音?只需一个样本!

Wanna hear your voice? A sample is all we need!

The Hieu Pham, Phuong Thanh Tran Nguyen, Xuan Tho Nguyen, Tan Dat Nguyen, Duc Dung Nguyen

AI总结:

针对低资源语言目标说话人提取的数据不足问题,提出 WHYV 跨语言零样本框架,通过调频门控机制降低语言依赖,并在英语与越南语数据集上取得先进性能。

摘要:

基于音频线索的目标说话人提取(TSE)研究一直聚焦于混合语音与参考语音建模,并因数据集丰富而在英语上取得优异结果。然而,跨语言特性仍未得到充分探索,因为低资源语言面临标注数据和语言资源有限的挑战。为弥合这一差距,我们提出 WHYV(Wanna Hear Your Voice),这是一种无需微调即可实现零样本适配的跨语言 TSE 框架。WHYV 采用调频门控机制,动态调整目标说话人的声学特征,从而尽量减少对语言特定线索的依赖。评估表明其具有最先进的零样本性能:在 Libri2Mix mix-both 上为 13.8 dB,在 mix-clean 上为 18.1 dB,并在越南语数据上达到 14.8 dB。

英文摘要:

Research on audio clue-based target speaker extraction (TSE) has focused on modeling mixtures and reference speech, achieving strong results in English due to abundant datasets. However, cross-lingual properties remain underexplored, as low-resource languages face challenges from limited annotated data and linguistic resources. To bridge this gap, we propose WHYV (Wanna Hear Your Voice), a cross-lingual TSE framework enabling zero-shot adaptation without fine-tuning. WHYV employs a frequency-modulated gating mechanism that dynamically adjusts the acoustic features of the target speaker, minimizing reliance on language-specific cues. Evaluations demonstrate state-of-the-art zero-shot performance: 13.8 dB (Libri2Mix mix-both), 18.1 dB (mix-clean), and 14.8 dB on Vietnamese data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑