GroupReduce：神经语言模型收缩的分块低秩近似
GroupReduce: Block-Wise Low-Rank Approximation for Neural Language Model Shrinking
摘要：
模型压缩对于在资源有限的设备或需要实时响应的应用程序上服务大型深度神经网络至关重要。作为一个案例研究，最先进的神经语言模型通常由一个或多个递归层组成，这些递归层夹在用于表示输入令牌的嵌入层和用于生成输出令牌的softmax层之间。对于词汇量非常大的问题，嵌入和softmax矩阵可以占模型大小的一半以上。例如，bigLSTM模型在拥有约80万词汇的十亿字（OBW）数据集上实现了最先进的性能，其词嵌入和softmax矩阵使用超过6Gbit空间，并负责超过90%的模型参数。在本文中，我们提出了GroupReduce，神经语言模型的一种新的压缩方法，基于词汇分区（块）的低秩矩阵近似和令牌的固有频率分布（词的幂律分布）。实验结果表明，该方法的压缩性能明显优于传统的压缩方法，如低秩近似和修剪。在OBW数据集上，我们的方法对嵌入和softmax矩阵实现了6.6倍的压缩率，当与量化相结合时，我们的方法可以实现26倍的压缩率，这意味着整个模型的压缩系数为12.8倍，并且在困惑度方面几乎没有退化。
英文摘要：
Model compression is essential for serving large deep neural nets on devices with limited resources or applications that require real-time responses. As a case study, a state-of-the-art neural language model usually consists of one or more recurrent layers sandwiched between an embedding layer used for representing input tokens and a softmax layer for generating output tokens. For problems with a very large vocabulary size, the embedding and the softmax matrices can account for more than half of the model size. For instance, the bigLSTM model achieves state-of- the-art performance on the One-Billion-Word (OBW) dataset with around 800k vocabulary, and its word embedding and softmax matrices use more than 6GBytes space, and are responsible for over 90% of the model parameters. In this paper, we propose GroupReduce, a novel compression method for neural language models, based on vocabulary-partition (block) based low-rank matrix approximation and the inherent frequency distribution of tokens (the power-law distribution of words). The experimental results show our method can significantly outperform traditional compression methods such as low-rank approximation and pruning. On the OBW dataset, our method achieved 6.6 times compression rate for the embedding and softmax matrices, and when combined with quantization, our method can achieve 26 times compression rate, which translates to a factor of 12.8 times compression for the entire model with very little degradation in perplexity.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
