基于稀疏随机三值矩阵的一维时间信道可分离卷积压缩
Compressing 1D Time-Channel Separable Convolutions using Sparse Random Ternary Matrices
摘要：
我们证明了一维时间通道可分离卷积中的1 × 1卷积可以用$\{-1,0,+1\}$中的带权重的稀疏随机三进制矩阵来代替。这样的层不执行任何乘法并且不需要训练。此外，矩阵可以在计算期间在芯片上生成，因此不需要任何存储器访问。在相同的参数预算下，我们可以提供更深入和更具表现力的模型，改善现有模型在几个任务上的帕累托边界。对于Google Speech Commands v1上的命令识别，我们在相同的网络大小下将最先进的准确性从$97.21\%$提高到了$97.41\%$。或者，我们可以降低现有模型的成本。对于Librispeech上的语音识别，我们将训练的权重数量减半，而只牺牲了浮点基线单词出错率的$1\%$。
英文摘要：
We demonstrate that 1x1-convolutions in 1D time-channel separable convolutions may be replaced by constant, sparse random ternary matrices with weights in $\{-1,0,+1\}$. Such layers do not perform any multiplications and do not require training. Moreover, the matrices may be generated on the chip during computation and therefore do not require any memory access. With the same parameter budget, we can afford deeper and more expressive models, improving the Pareto frontiers of existing models on several tasks. For command recognition on Google Speech Commands v1, we improve the state-of-the-art accuracy from $97.21\%$ to $97.41\%$ at the same network size. Alternatively, we can lower the cost of existing models. For speech recognition on Librispeech, we half the number of weights to be trained while only sacrificing about $1\%$ of the floating-point baseline's word error rate.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
