arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1906.04886cs.LGcs.NEstat.ML
收藏

用于边缘设备推理的运行时间高效RNN压缩

Run-Time Efficient RNN Compression for Inference on Edge Devices

Arm ML Research(Arm机器学习研究院)

Urmish Thakker, Jesse Beu, Dibakar Gope, Ganesh Dasika, Matthew Mattina

摘要:

递归神经网络可能是大型的,计算密集型的,但许多受益于RNN的应用程序运行在计算和存储能力非常有限的小型设备上,同时仍然具有运行时间限制。因此,需要能够实现显著压缩而不负面影响推理运行时间和任务准确性的压缩技术。本文探讨了一种新的压缩RNN单元实现,称为混合矩阵分解(HMD),可以实现这一双重目标。该方案将权重矩阵分为两部分-无约束的上半部分和由秩1块组成的下半部分。这导致输出特征,其中上部子向量具有“更丰富”的特征,而下部子向量具有“受约束的特征”。HMD可以将RNN压缩2- 4倍,同时具有比修剪更快的运行时间(Zhu &Gupta,2017),并保留比矩阵分解更高的模型准确性(Grachev等人,2017年)的报告。我们在跨越3个不同应用的5个基准测试中评估了这项技术,说明了它在边缘计算领域的通用性。

英文摘要:

Recurrent neural networks can be large and compute-intensive, yet many applications that benefit from RNNs run on small devices with very limited compute and storage capabilities while still having run-time constraints. As a result, there is a need for compression techniques that can achieve significant compression without negatively impacting inference run-time and task accuracy. This paper explores a new compressed RNN cell implementation called Hybrid Matrix Decomposition (HMD) that achieves this dual objective. This scheme divides the weight matrix into two parts - an unconstrained upper half and a lower half composed of rank-1 blocks. This results in output features where the upper sub-vector has "richer" features while the lower-sub vector has "constrained features". HMD can compress RNNs by a factor of 2-4x while having a faster run-time than pruning (Zhu &Gupta, 2017) and retaining more model accuracy than matrix factorization (Grachev et al., 2017). We evaluate this technique on 5 benchmarks spanning 3 different applications, illustrating its generality in the domain of edge computing.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑