arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2210.15324eess.AScs.SD
收藏

鲁棒的Data2vec:结合回归和改进对比学习的ASR抗噪声语音表示学习

Robust Data2vec: Noise-robust Speech Representation Learning for ASR by Combining Regression and Improved Contrastive Learning

University of Science and Technology of China(中国科学技术大学);Microsoft Research Asia(微软亚洲研究院);Nanyang Technological University(南洋理工大学)

Qiu-Shi Zhu, Long Zhou, Jie Zhang, Shu-Jie Liu, Yu-Chen Hu, Li-Rong Dai

摘要:

基于对比学习或回归任务的自监督预训练方法可以利用更多的未标记数据来提高自动语音识别(ASR)的性能。然而,结合两个预训练任务和构建不同的负样本进行对比学习的鲁棒性影响仍然不清楚。在本文中,我们提出了一种用于自监督语音表示学习的噪声鲁棒data 2 vec,通过在预训练阶段联合优化对比学习和回归任务。此外,我们提出了两种改进的方法,以促进对比学习。更具体地说,我们首先提出构建基于块的非语义负样本来提高预训练模型的噪声鲁棒性,这是通过将特征划分为不同大小的块来实现的(即,所谓的负样本)。其次,通过分析正样本和负样本的分布,我们提出了去除容易区分的负样本,以提高预训练模型的区分能力。在CHiME-4数据集上的实验结果表明,我们的方法能够提高噪声场景中预训练模型的性能。我们发现,对比学习和回归任务的联合训练可以在一定程度上避免模型崩溃相比,只训练回归任务。

英文摘要:

Self-supervised pre-training methods based on contrastive learning or regression tasks can utilize more unlabeled data to improve the performance of automatic speech recognition (ASR). However, the robustness impact of combining the two pre-training tasks and constructing different negative samples for contrastive learning still remains unclear. In this paper, we propose a noise-robust data2vec for self-supervised speech representation learning by jointly optimizing the contrastive learning and regression tasks in the pre-training stage. Furthermore, we present two improved methods to facilitate contrastive learning. More specifically, we first propose to construct patch-based non-semantic negative samples to boost the noise robustness of the pre-training model, which is achieved by dividing the features into patches at different sizes (i.e., so-called negative samples). Second, by analyzing the distribution of positive and negative samples, we propose to remove the easily distinguishable negative samples to improve the discriminative capacity for pre-training models. Experimental results on the CHiME-4 dataset show that our method is able to improve the performance of the pre-trained model in noisy scenarios. We find that joint training of the contrastive learning and regression tasks can avoid the model collapse to some extent compared to only training the regression task.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑