用于LVCSR的极深卷积神经网络的进展
Advances in Very Deep Convolutional Neural Networks for LVCSR
摘要：
具有小的3x 3内核的非常深的CNN最近被证明在混合NN-HMM语音识别系统中作为声学模型实现非常强的性能。在本文中，我们将研究如何有效地将这些模型扩展到更大的数据集。具体来说，我们解决了沿着时间维度的池化和填充的设计选择，这使得序列的卷积评估非常低效。我们提出了一种新的CNN设计，没有时间填充和时间池，这是稍微次优的准确性，但有两个显着的优势：它使序列训练和部署，允许有效的卷积评估的完整的话语，它允许批量归一化直接采用CNN的序列数据。通过批量归一化，我们恢复了由于删除时间池而丢失的概率，同时保持了高效卷积计算的好处。我们展示了我们的模型在比以前更大规模的数据上以及在序列训练之后的性能。我们在2000 h交换机数据集上训练的非常深的CNN模型序列在Hub 5测试集上获得了9.4字的出错率，与2015 IBM系统组合的性能相匹配，这是之前公布的最佳结果。
英文摘要：
Very deep CNNs with small 3x3 kernels have recently been shown to achieve very strong performance as acoustic models in hybrid NN-HMM speech recognition systems. In this paper we investigate how to efficiently scale these models to larger datasets. Specifically, we address the design choice of pooling and padding along the time dimension which renders convolutional evaluation of sequences highly inefficient. We propose a new CNN design without timepadding and without timepooling, which is slightly suboptimal for accuracy, but has two significant advantages: it enables sequence training and deployment by allowing efficient convolutional evaluation of full utterances, and, it allows for batch normalization to be straightforwardly adopted to CNNs on sequence data. Through batch normalization, we recover the lost peformance from removing the time-pooling, while keeping the benefit of efficient convolutional evaluation. We demonstrate the performance of our models both on larger scale data than before, and after sequence training. Our very deep CNN model sequence trained on the 2000h switchboard dataset obtains 9.4 word error rate on the Hub5 test-set, matching with a single model the performance of the 2015 IBM system combination, which was the previous best published result.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
