arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2007.06631cs.LGcs.CVstat.ML
收藏

T-Basis:神经网络的一种紧凑表示

T-Basis: a Compact Representation for Neural Networks

ETH Zurich(苏黎世联邦理工学院);KU Leuven(鲁汶大学)

Anton Obukhov, Maxim Rakhuba, Stamatios Georgoulis, Menelaos Kanakis, Dengxin Dai, Luc Van Gool

摘要:

我们介绍了T基,一个新的概念,一组张量的紧凑表示,每个任意形状,这是经常看到的神经网络。集合中的每个张量都是使用张量环建模的,尽管这个概念适用于其他张量网络。由于它的名字来自张量环图符号中的T形节点,T基只是一个相等形状的三维张量列表,用于表示张量环节点。这样的表示允许我们用少量的参数(T基张量的系数)来参数化张量集,随着集合中每个张量的大小进行几何缩放,并与T基的维度进行线性缩放。我们评估所提出的方法的任务上的神经网络压缩,并证明它达到高压缩率在可接受的性能下降。最后,我们分析了压缩网络的内存和操作需求,并得出结论,T-Basis网络同样适合在资源受限的环境中进行训练和推理,并在边缘设备上使用。

英文摘要:

We introduce T-Basis, a novel concept for a compact representation of a set of tensors, each of an arbitrary shape, which is often seen in Neural Networks. Each of the tensors in the set is modeled using Tensor Rings, though the concept applies to other Tensor Networks. Owing its name to the T-shape of nodes in diagram notation of Tensor Rings, T-Basis is simply a list of equally shaped three-dimensional tensors, used to represent Tensor Ring nodes. Such representation allows us to parameterize the tensor set with a small number of parameters (coefficients of the T-Basis tensors), scaling logarithmically with each tensor's size in the set and linearly with the dimensionality of T-Basis. We evaluate the proposed approach on the task of neural network compression and demonstrate that it reaches high compression rates at acceptable performance drops. Finally, we analyze memory and operation requirements of the compressed networks and conclude that T-Basis networks are equally well suited for training and inference in resource-constrained environments and usage on the edge devices.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑