conv_einsum：卷积张量神经网络中多重线性操作的表示与快速评估框架
conv_einsum: A Framework for Representation and Fast Evaluation of Multilinear Operations in Convolutional Tensorial Neural Networks
AI总结：
本文提出统一框架将张量卷积层表示为类einsum字符串，并开发元算法conv_einsum以最小化FLOPs方式评估，显著提升了卷积TNNs的计算与内存效率。
摘要：
现代 ConvNets 在大量视觉和图像分类任务上持续取得最先进的结果，但代价是参数量不断增加。一种在不牺牲太多表达能力的情况下使网络紧凑化的策略是将其重塑为张量神经网络（TNN），这是对其层的更高阶张量化，随后进行分解（例如 CP-decomposition），将权重精简至其关键基组件。通过 TNNs 的前向传播可以表示为多重线性操作（MLOs）序列，其中评估路径会极大影响所引发的浮点运算次数（FLOPs）。虽然像流行的 einsum 这样的函数可以评估诸如收缩等简单的 MLOs，但现有的实现无法处理多路卷积，导致对通过张量化卷积层的最优评估路径如何提高训练速度缺乏充分评估。在本文中，我们开发了一个统一框架，将张量卷积层表示为类似 einsum 的字符串，并提出了一种元算法 conv_einsum，它能够以最小化 FLOPs 的方式评估这些字符串。使用我们的开源实现，在广泛的模型、张量分解和多样化任务上进行的综合实验表明，conv_einsum 显著提高了卷积 TNNs 的计算和内存效率。
英文摘要：
Modern ConvNets continue to achieve state-of-the-art results over a vast array of vision and image classification tasks, but at the cost of increasing parameters. One strategy for compactifying a network without sacrificing much expressive power is to reshape it into a tensorial neural network (TNN), which is a higher-order tensorization of its layers, followed by a factorization, such as a CP-decomposition, which strips a weight down to its critical basis components. Passes through TNNs can be represented as sequences of multilinear operations (MLOs), where the evaluation path can greatly affect the number of floating point operations (FLOPs) incurred. While functions such as the popular einsum can evaluate simple MLOs such as contractions, existing implementations cannot process multi-way convolutions, resulting in scant assessments of how optimal evaluation paths through tensorized convolutional layers can improve training speed. In this paper, we develop a unifying framework for representing tensorial convolution layers as einsum-like strings and a meta-algorithm conv_einsum which is able to evaluate these strings in a FLOPs-minimizing manner. Comprehensive experiments, using our open-source implementation, over a wide range of models, tensor decompositions, and diverse tasks, demonstrate that conv_einsum significantly increases both computational and memory-efficiency of convolutional TNNs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
