ProdSumNet：通过和积矩阵分解减少深度神经网络中的模型参数
ProdSumNet: reducing model parameters in deep neural networks via product-of-sums matrix decompositions
摘要：
我们考虑一个通用框架，通过将线性算子分解为简单线性算子之和的乘积，来减少深度学习网络中可训练模型参数的数量。最近提出的深度学习架构，如CNN，KFC，扩张的CNN等都包含在这个框架中，我们在这个框架内说明了其他类型的神经网络架构。我们表明，MNIST和时尚MNIST的良好的准确性，可以使用相对较少的可训练参数。此外，由于卷积层的实现是资源密集型的，因此我们考虑在变换域中采用一种方法来消除对卷积层的需求。这种一般框架优于现有方法的优点之一是可训练参数的数量不是固定的，并且可以任意变化。特别是，我们说明了不同的可训练变量的数量和相应的出错率的权衡。作为一个例子，通过在具有超过3x10^6个可训练参数的MNIST的参考CNN架构上使用这种分解，我们能够仅使用3554个可训练参数获得98.44%的准确度。
英文摘要：
We consider a general framework for reducing the number of trainable model parameters in deep learning networks by decomposing linear operators as a product of sums of simpler linear operators. Recently proposed deep learning architectures such as CNN, KFC, Dilated CNN, etc. are all subsumed in this framework and we illustrate other types of neural network architectures within this framework. We show that good accuracy on MNIST and Fashion MNIST can be obtained using a relatively small number of trainable parameters. In addition, since implementation of the convolutional layer is resource-heavy, we consider an approach in the transform domain that obviates the need for convolutional layers. One of the advantages of this general framework over prior approaches is that the number of trainable parameters is not fixed and can be varied arbitrarily. In particular, we illustrate the tradeoff of varying the number of trainable variables and the corresponding error rate. As an example, by using this decomposition on a reference CNN architecture for MNIST with over 3x10^6 trainable parameters, we are able to obtain an accuracy of 98.44% using only 3554 trainable parameters.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
