arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1809.06833eess.AS
收藏

使用领域特定的师生学习范式推进多重音LSTM-CTC语音识别

Advancing Multi-Accented LSTM-CTC Speech Recognition using a Domain Specific Student-Teacher Learning Paradigm

Shahram Ghorbani, Ahmet E. Bulut, John H. L. Hansen

摘要:

非母语语音会导致自动语音识别系统的性能下降。过去的策略,以解决这一挑战,考虑模型自适应,口音分类与模型选择,替代发音词典等,在这项研究中,我们考虑了一个循环神经网络(RNN)与连接时间分类(CTC)代价函数训练的多口音英语数据,包括美国(本地),印度和西班牙口音。我们利用暗知识从多口音数据训练的模型训练学生模型的指导下,教师模型和目标转录的CTC成本。我们表明,将知识从单个RNN-CTC训练模型转移到学生模型,比独立的教师模型产生更好的性能。由于不同训练的CTC模型的输出不一定一致,因此不可能简单地使用CTC教师模型的集合。为了解决这个问题,我们在单个多口音教师的指导下训练口音特定模型,这导致具有多个对齐和训练的CTC模型。此外,我们在特定口音教师的监督下训练了一个学生模型,从而产生了一个更进一步的补充模型,与没有任何教师训练的基线相比,该模型实现了+20.1%的相对字符错误率(CER)降低。有了这个有效的多口音模型,我们可以通过使模型适应每种口音来进一步改进每种口音。使用重音特定模型的输出来正则化适配过程(即,Kullback-Leibler(KL)发散的知识蒸馏版本)与使用一般教师模型的常规方法相比甚至产生更好的性能。

英文摘要:

Non-native speech causes automatic speech recognition systems to degrade in performance. Past strategies to address this challenge have considered model adaptation, accent classification with a model selection, alternate pronunciation lexicon, etc. In this study, we consider a recurrent neural network (RNN) with connectionist temporal classification (CTC) cost function trained on multi-accent English data including US (Native), Indian and Hispanic accents. We exploit dark knowledge from a model trained with the multi-accent data to train student models under the guidance of both a teacher model and CTC cost of target transcription. We show that transferring knowledge from a single RNN-CTC trained model toward a student model, yields better performance than the stand-alone teacher model. Since the outputs of different trained CTC models are not necessarily aligned, it is not possible to simply use an ensemble of CTC teacher models. To address this problem, we train accent specific models under the guidance of a single multi-accent teacher, which results in having multiple aligned and trained CTC models. Furthermore, we train a student model under the supervision of the accent-specific teachers, resulting in an even further complementary model, which achieves +20.1% relative Character Error Rate (CER) reduction compared to the baseline trained without any teacher. Having this effective multi-accent model, we can achieve further improvement for each accent by adapting the model to each accent. Using the accent specific model's outputs to regularize the adapting process (i.e., a knowledge distillation version of Kullback-Leibler (KL) divergence) results in even superior performance compared to the conventional approach using general teacher models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑