Bayesian Compression for Natural Language Processing
发表机构国立研究型大学高等经济学院 · 三星人工智能中心
查看机构详情
- National Research University Higher School of Economics（国立研究型大学高等经济学院）
- Samsung AI Center（三星人工智能中心）
机构由 AI 辅助整理，请以论文原文为准。
首次发表 更新
浏览论文内容
英文摘要
In natural language processing, a lot of the tasks are successfully solved with recurrent neural networks, but such models have a huge number of parameters. The majority of these parameters are often concentrated in the embedding layer, which size grows proportionally to the vocabulary length. We propose a Bayesian sparsification technique for RNNs which allows compressing the RNN dozens or hundreds of times without time-consuming hyperparameters tuning. We also generalize the model for vocabulary sparsification to filter out unnecessary words and compress the RNN even further. We show that the choice of the kept words is interpretable. Code is available on github: https://github.com/tipt0p/SparseBayesianRNN
