arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2102.02326cs.LG
收藏

卷积层滤波器数目对语音识别模型精度的影响

Effects of Number of Filters of Convolutional Layers on Speech Recognition Model Accuracy

Strategical Solutions, OptumCare(OptumCare战略解决方案);UnitedHealth Group(联合健康集团)

James Mou, Jun Li

摘要:

受端到端方法[1]进展的启发,本文系统地研究了卷积层滤波器数目对CNN+RNN(Convolutional Neural Networks adding to Recurrent Neural Networks,卷积神经网络加递归神经网络)ASR模型(Automatic语音识别)预测精度的影响。实验结果表明,只有当CNN Number of Filters超过一定的阈值时,将CNN加入RNN才能提高CNN+RNN语音识别模型的性能,否则CNN的某些参数范围会使将CNN加入RNN模型变得无用。我们的研究结果表明,单词准确性对卷积层的过滤器数量有很强的依赖性。基于实验结果,本文提出了一种可能的声-2-向量嵌入(卷积嵌入)假说来解释上述现象。基于这一嵌入假设和参数优化,本文开发了一个端到端的语音识别系统,具有较高的词的准确率,但也有一个轻的模型重量。所开发的大词汇量连续语音识别(LVCSR)模型,在没有中间语音表示和任何语言模型辅助的情况下,仅依靠声学模型就取得了90.2%的较高的词识别率。其声学模型仅包含440万个权重参数,而DeepSpeech 2 [2](最先进的LVCSR模型之一)中的声学模型权重参数为3500万至6800万,可以实现91.5%的单词准确率。轻量级模型有利于提高转录计算效率,也适用于移动设备,无人驾驶汽车等。我们的模型重量减少到DeepSpeech 2大小的10%左右,但我们的模型精度仍然接近DeepSpeech 2。如果结合语言模型,我们的LVCSR系统能够达到91.5%的单词准确率。

英文摘要:

Inspired by the progress of the End-to-End approach [1], this paper systematically studies the effects of Number of Filters of convolutional layers on the model prediction accuracy of CNN+RNN (Convolutional Neural Networks adding to Recurrent Neural Networks) for ASR Models (Automatic Speech Recognition). Experimental results show that only when the CNN Number of Filters exceeds a certain threshold value is adding CNN to RNN able to improve the performance of the CNN+RNN speech recognition model, otherwise some parameter ranges of CNN can render it useless to add the CNN to the RNN model. Our results show a strong dependency of word accuracy on the Number of Filters of convolutional layers. Based on the experimental results, the paper suggests a possible hypothesis of Sound-2-Vector Embedding (Convolutional Embedding) to explain the above observations. Based on this Embedding hypothesis and the optimization of parameters, the paper develops an End-to-End speech recognition system which has a high word accuracy but also has a light model-weight. The developed LVCSR (Large Vocabulary Continuous Speech Recognition) model has achieved quite a high word accuracy of 90.2% only by its Acoustic Model alone, without any assistance from intermediate phonetic representation and any Language Model. Its acoustic model contains only 4.4 million weight parameters, compared to the 35~68 million acoustic-model weight parameters in DeepSpeech2 [2] (one of the top state-of-the-art LVCSR models) which can achieve a word accuracy of 91.5%. The light-weighted model is good for improving the transcribing computing efficiency and also useful for mobile devices, Driverless Vehicles, etc. Our model weight is reduced to ~10% the size of DeepSpeech2, but our model accuracy remains close to that of DeepSpeech2. If combined with a Language Model, our LVCSR system is able to achieve 91.5% word accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑