PLUM:利用重复-稀疏性权衡提升推理效率
PLUM: Improving Inference Efficiency By Leveraging Repetition-Sparsity Trade-Off
AI总结:
本文提出PLUM统一协同设计框架,利用重复-稀疏性权衡优化DNN推理,在保持准确率的同时实现26%加速和2.8倍密度降低。
摘要:
在资源受限的边缘设备上高效推理深度神经网络(DNNs)至关重要。量化与稀疏性是两项关键技术,它们在硬件-软件接口处转化为张量内部的重复与稀疏。本文引入了重复-稀疏性权衡的概念,有助于解释推理过程中的计算效率。我们提出了PLUM,一个统一的协同设计框架,将DNN推理系统与量化(前向和反向传播)集成在一起,以利用重复-稀疏性权衡来提升推理效率。我们的结果表明,在非零权重数量相同的情况下,PLUM的量化方法比二值量化更准确。详细分析表明,有符号二值化在一个DNN块的潜在全精度权重的较大总参数分布中,生成了一个嵌套在其中的较小的有效(非零)参数分布。最后,所提出的PLUM框架在真实硬件上实现了26%的加速,能效提升一倍,与二值方法相比密度降低了2.8倍,同时在ImageNet上的ResNets与先前最优方法相比保持了top-1准确率(达到66.2%的top-1准确率),为在资源有限的环境中部署高效模型提供了一种替代解决方案。
英文摘要:
Efficient inference of Deep Neural Networks (DNNs) on resource-constrained edge devices is essential. Quantization and sparsity are key techniques that translate to repetition and sparsity within tensors at the hardware-software interface. This paper introduces the concept of repetition-sparsity trade-off that helps explain computational efficiency during inference. We propose PLUM, a unified co-design framework that integrates DNN inference systems and quantization (forward and backward pass) to leverage the repetition-sparsity trade-off to improve inference efficiency. Our results demonstrate that PLUM's quantization method is more accurate than binary quantization with the same number of non-zero weights. Detailed analysis indicates that signed binarization generates a smaller distribution of effectual (non-zero) parameters nested within a larger distribution of total parameters of latent full-precision weights for a DNN block. Finally, the proposed PLUM framework achieves a 26% speedup on real hardware, doubles energy efficiency, and reduces density by 2.8x compared to binary methods while retaining top-1 accuracy when compared to prior-art methods for ResNets on ImageNet (by achieving 66.2% top-1 accuracy), presenting an alternative solution for deploying efficient models in resource-limited environments.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。