利用WTA哈希算法加速神经网络的大规模分类
Speeding Up Neural Networks for Large Scale Classification using WTA Hashing
摘要：
在本文中，我们提出了使用Winner Takes All散列技术，以加快在卷积神经网络的全连接层的前向传播和后向传播。所提出的技术显着降低了计算复杂性，这反过来又使我们能够训练层与大量的内核，而没有相关的时间惩罚。因此，我们能够在非常大量的输出类上训练卷积神经网络，而计算成本只有很小的增加。为了证明该技术的有效性，我们使用常规乘法方法和我们提出的哈希方法在预先训练的网络上训练一个新的输出层。我们的结果显示性能没有下降，并且通过我们的实现，在训练期间速度提高了7倍。
英文摘要：
In this paper we propose to use the Winner Takes All hashing technique to speed up forward propagation and backward propagation in fully connected layers in convolutional neural networks. The proposed technique reduces significantly the computational complexity, which in turn, allows us to train layers with a large number of kernels with out the associated time penalty. As a consequence we are able to train convolutional neural network on a very large number of output classes with only a small increase in the computational cost. To show the effectiveness of the technique we train a new output layer on a pretrained network using both the regular multiplicative approach and our proposed hashing methodology. Our results showed no drop in performance and demonstrate, with our implementation, a 7 fold speed up during the training.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
