arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1509.06569cs.LGcs.NE
收藏

张量化神经网络

Tensorizing Neural Networks

Skolkovo Institute of Science and Technology(斯科尔科沃科学技术研究院);INRIA(法国国家信息与自动化研究所);National Research University Higher School of Economics(国立研究型大学高等经济学院);Institute of Numerical Mathematics of the Russian Academy of Sciences(俄罗斯科学院数值数学研究所)

Alexander Novikov, Dmitry Podoprikhin, Anton Osokin, Dmitry Vetrov

摘要:

深度神经网络目前在几个领域表现出最先进的性能。同时,这类模型在计算资源方面要求非常高。特别是,常用的全连接层需要大量的内存,这使得很难在低端设备上使用模型,并阻止模型大小的进一步增加。在本文中,我们将全连接层的稠密权重矩阵转换为张量训练格式,使得参数的数量减少了一个巨大的因素,同时保留了层的表达能力。特别是,对于非常深的VGG网络,我们报告了全连接层的密集权重矩阵的压缩因子高达200000倍,导致整个网络的压缩因子高达7倍。

英文摘要:

Deep neural networks currently demonstrate state-of-the-art performance in several domains. At the same time, models of this class are very demanding in terms of computational resources. In particular, a large amount of memory is required by commonly used fully-connected layers, making it hard to use the models on low-end devices and stopping the further increase of the model size. In this paper we convert the dense weight matrices of the fully-connected layers to the Tensor Train format such that the number of parameters is reduced by a huge factor and at the same time the expressive power of the layer is preserved. In particular, for the Very Deep VGG networks we report the compression factor of the dense weight matrix of a fully-connected layer up to 200000 times leading to the compression factor of the whole network up to 7 times.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑