arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2104.01471eess.AS
收藏

基于自注意机制的对抗性联合训练的鲁棒端到端语音识别

Adversarial Joint Training with Self-Attention Mechanism for Robust End-to-End Speech Recognition

Lujun Li, Yikai Kang, Yuchen Shi, Ludwig Kürzinger, Tobias Watzel, Gerhard Rigoll

摘要:

近年来,自注意机制成为自动语音识别(ASR)领域的一个新的里程碑。然而,它的性能容易受到环境入侵的影响,因为系统根据完整的输入序列和先前的预测来预测下一个输出符号。受生成对抗网络(GANs)在语音增强和ASR任务中广泛应用的启发,提出了一种带有自注意机制的对抗联合训练框架,以提高ASR系统的噪声鲁棒性。通常,它由自注意语音增强GAN和自注意端到端ASR模型组成。在这一拟议框架中,有两个重点值得注意。一是得益于自注意机制和GANs的进步;二是GAN的泛化能力在对抗性联合训练阶段扮演全局判别网络的角色,引导增强前端为后续的ASR模块捕获更多兼容的结构,从而弥补了单独训练和手工制作损失函数的局限性。通过对抗性联合优化,该框架有望学习更适合ASR任务的鲁棒表示。在AISHELL-1语料库上进行了系统的实验,实验结果表明,在人工噪声测试集上,该框架与单纯使用干净数据训练的ASR模型相比,相对性能提高了66%,与不使用联合训练的语音增强和ASR方案相比,相对性能提高了35.1%,与多条件训练相比,相对性能提高了5.3%。

英文摘要:

Lately, the self-attention mechanism has marked a new milestone in the field of automatic speech recognition (ASR). Nevertheless, its performance is susceptible to environmental intrusions as the system predicts the next output symbol depending on the full input sequence and the previous predictions. Inspired by the extensive applications of the generative adversarial networks (GANs) in speech enhancement and ASR tasks, we propose an adversarial joint training framework with the self-attention mechanism to boost the noise robustness of the ASR system. Generally, it consists of a self-attention speech enhancement GAN and a self-attention end-to-end ASR model. There are two highlights which are worth noting in this proposed framework. One is that it benefits from the advancement of both self-attention mechanism and GANs; while the other is that the discriminator of GAN plays the role of the global discriminant network in the stage of the adversarial joint training, which guides the enhancement front-end to capture more compatible structures for the subsequent ASR module and thereby offsets the limitation of the separate training and handcrafted loss functions. With the adversarial joint optimization, the proposed framework is expected to learn more robust representations suitable for the ASR task. We execute systematic experiments on the corpus AISHELL-1, and the experimental results show that on the artificial noisy test set, the proposed framework achieves the relative improvements of 66% compared to the ASR model trained by clean data solely, 35.1% compared to the speech enhancement & ASR scheme without joint training, and 5.3% compared to multi-condition training.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑