可分离的层实现结构化的高效线性替换
Separable Layers Enable Structured Efficient Linear Substitutions
摘要：
为了响应最近高效密集层的发展，本文表明，像用结构化线性分解替换逐点卷积中的线性分量这样简单的事情也会在效率/精度权衡方面产生实质性的收益。点态卷积是完全连接的层，因此准备好被结构化变换替换。使用这些层的网络能够学习与使用标准卷积的网络相同的任务，并在计算（多加）和参数计数（以及内存）方面提供效率/准确性方面的帕累托最优优势。代码可在https://github.com/BayesWatch/deficient-efficient上获得。
英文摘要：
In response to the development of recent efficient dense layers, this paper shows that something as simple as replacing linear components in pointwise convolutions with structured linear decompositions also produces substantial gains in the efficiency/accuracy tradeoff. Pointwise convolutions are fully connected layers and are thus prepared for replacement by structured transforms. Networks using such layers are able to learn the same tasks as those using standard convolutions, and provide Pareto-optimal benefits in efficiency/accuracy, both in terms of computation (mult-adds) and parameter count (and hence memory). Code is available at https://github.com/BayesWatch/deficient-efficient.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
