使用哈希技巧压缩神经网络
Compressing Neural Networks with the Hashing Trick
摘要：
随着深度网络越来越多地用于适合移动设备的应用程序中，一个基本的困境变得明显：深度学习的趋势是增加模型以吸收不断增加的数据集大小;然而，移动设备的内存设计很少，无法存储如此大的模型。我们提出了一种新的网络架构，哈希网络，利用固有的冗余神经网络，以实现大幅减少模型的大小。HashedNets使用低成本的哈希函数将连接权重随机分组到哈希桶中，同一哈希桶中的所有连接共享一个参数值。这些参数经过调整，以适应训练期间具有标准反向传播的HashedNets权重共享架构。我们的散列过程不会引入额外的内存开销，并且我们在几个基准数据集上证明了HashedNets大幅缩减了神经网络的存储需求，同时大部分保留了泛化性能。
英文摘要：
As deep nets are increasingly used in applications suited for mobile devices, a fundamental dilemma becomes apparent: the trend in deep learning is to grow models to absorb ever-increasing data set sizes; however mobile devices are designed with very little memory and cannot store such large models. We present a novel network architecture, HashedNets, that exploits inherent redundancy in neural networks to achieve drastic reductions in model sizes. HashedNets uses a low-cost hash function to randomly group connection weights into hash buckets, and all connections within the same hash bucket share a single parameter value. These parameters are tuned to adjust to the HashedNets weight sharing architecture with standard backprop during training. Our hashing procedure introduces no additional memory overhead, and we demonstrate on several benchmark data sets that HashedNets shrink the storage requirements of neural networks substantially while mostly preserving generalization performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
