arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.02565eess.AScs.SD
收藏

从自监督语音模型中高效提取抗噪离散单元

Efficient Extraction of Noise-Robust Discrete Units from Self-Supervised Speech Models

KU Leuven(鲁汶大学);Xiaomi Corporation(小米公司)

Jakob Poncelet, Yujun Wang, Hugo Van hamme

AI总结:

本文提出一种参数高效的小型编码器-解码器方法,对自监督语音模型隐藏特征同时去噪和离散化,以生成抗噪离散单元,在带噪离散化和语音识别任务上优于多种预训练方法,并可用少量无标签数据微调适应目标环境。

摘要:

连续语音可以通过从自监督学习(SSL)语音模型的隐藏特征中推导离散单元而转换为离散序列。尽管SSL模型变得越来越大并在更多数据上训练,它们通常对现实生活中的失真(如加性噪声或混响)敏感,这会导致离散单元发生偏移。我们提出一种参数高效的方法,通过训练一个小型编码器-解码器模型(带或不带适配器),从预训练的SSL模型中生成抗噪离散单元,同时对其隐藏特征进行去噪和离散化。该模型学习在SSL特征的条件下,为带噪语音生成干净的离散序列。所提出的去噪器在带噪离散化和带噪语音识别任务上优于多种预训练方法,并且可以利用少量无标签目标环境录音微调到目标环境。

英文摘要:

Continuous speech can be converted into a discrete sequence by deriving discrete units from the hidden features of self-supervised learned (SSL) speech models. Although SSL models are becoming larger and trained on more data, they are often sensitive to real-life distortions like additive noise or reverberation, which translates to a shift in discrete units. We propose a parameter-efficient approach to generate noise-robust discrete units from pre-trained SSL models by training a small encoder-decoder model, with or without adapters, to simultaneously denoise and discretise the hidden features of the SSL model. The model learns to generate a clean discrete sequence for a noisy utterance, conditioned on the SSL features. The proposed denoiser outperforms several pre-training methods on the tasks of noisy discretisation and noisy speech recognition, and can be finetuned to the target environment with a few recordings of unlabeled target data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑