arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1706.09453cs.CL
收藏

具有二元权值和激活的计算和存储效率高的神经网络声学模型

Toward Computation and Memory Efficient Neural Network Acoustic Models with Binary Weights and Activations

Toyota Technological Institute at Chicago(芝加哥丰田理工学院)

Liang Lu

摘要:

在过去的几年里,神经网络声学模型已经显著地推进了语音识别的技术发展水平。然而,由于大量的矩阵向量乘法和非线性运算,它们通常在计算上是昂贵的。神经网络模型也需要大量的内存用于推理,因为模型大小很大。由于这两个原因,在诸如嵌入式设备的资源受限的平台上部署基于神经网络的语音识别器是具有挑战性的。本文研究了使用二进制权重和激活的计算和存储效率的神经网络声学模型。与实值权重矩阵相比,二进制权重需要更少的位用于存储,从而减少了内存占用。此外,通过二进制权重或激活,矩阵向量乘法被转换为加法和减法运算,这对于硬件平台来说在计算上更快并且更节能。在本文中,我们研究的应用程序的二进制权重和激活神经网络声学建模,报告令人鼓舞的结果WSJ和AMI语料库。

英文摘要:

Neural network acoustic models have significantly advanced state of the art speech recognition over the past few years. However, they are usually computationally expensive due to the large number of matrix-vector multiplications and nonlinearity operations. Neural network models also require significant amounts of memory for inference because of the large model size. For these two reasons, it is challenging to deploy neural network based speech recognizers on resource-constrained platforms such as embedded devices. This paper investigates the use of binary weights and activations for computation and memory efficient neural network acoustic models. Compared to real-valued weight matrices, binary weights require much fewer bits for storage, thereby cutting down the memory footprint. Furthermore, with binary weights or activations, the matrix-vector multiplications are turned into addition and subtraction operations, which are computationally much faster and more energy efficient for hardware platforms. In this paper, we study the applications of binary weights and activations for neural network acoustic modeling, reporting encouraging results on the WSJ and AMI corpora.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑