深度学习推理中的稀疏计算
Sparse Computations in Deep Learning Inference
AI总结:
本文探讨了深度学习推理中稀疏计算的多种形式及其优化方法,旨在为性能工程师提供高效稀疏模型开发和部署的指导。
摘要:
现代深度神经网络(DNN)的计算需求巨大且持续增长。虽然训练成本通常引起公众关注,但推理需求也在显著的计算、能源和环境足迹中发挥重要作用。稀疏性突出作为大幅减少这些资源需求的关键机制。然而,其潜力仍 largely 未被充分利用,并未完全融入生产AI系统中。为弥合这一差距,本文提供了必要的知识和见解,以帮助性能工程师参与深度学习推理优化。特别是,本文讨论了可以在DNN推理中利用的各种稀疏性形式,解释了原始密集计算如何转化为稀疏内核,提供了这些内核在CPU和GPU上实现的最新文献综述,讨论了支持稀疏性研究和开发的稀疏数据集的可用性,探索了提供稳健稀疏性支持的当前软件工具和框架,并展示了不同实现的关键SpMM和SDDMM内核在CPU和GPU平台上的评估结果。最终,本文旨在为寻求开发和部署高效稀疏深度学习模型的性能工程师提供资源。
英文摘要:
The computational demands of modern Deep Neural Networks (DNNs) are immense and constantly growing. While training costs usually capture public attention, inference demands are also contributing in significant computational, energy and environmental footprints. Sparsity stands out as a critical mechanism for drastically reducing these resource demands. However, its potential remains largely untapped and is not yet fully incorporated in production AI systems. To bridge this gap, this work provides the necessary knowledge and insights for performance engineers keen to get involved in deep learning inference optimization. In particular, in this work we: a) discuss the various forms of sparsity that can be utilized in DNN inference, b) explain how the original dense computations translate to sparse kernels, c) provide an extensive bibliographic review of the state-of-the-art in the implementation of these kernels for CPUs and GPUs, d) discuss the availability of sparse datasets in support of sparsity-related research and development, e) explore the current software tools and frameworks that provide robust sparsity support, and f) present evaluation results of different implementations of the key SpMM and SDDMM kernels on CPU and GPU platforms. Ultimately, this paper aims to serve as a resource for performance engineers seeking to develop and deploy highly efficient sparse deep learning models in productions.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。