优化深度学习加速器的内存访问模式
Optimizing Memory-Access Patterns for Deep Learning Accelerators
摘要：
深度学习（DL）工作负载正朝着加速器方向发展，以实现更快的处理速度和更低的成本。现代DL加速器擅长处理主导DL工作负载的大规模乘法-累加运算;然而，充分利用加速器的计算能力是具有挑战性的，因为数据必须在软件管理的暂存器存储器中正确分级。如果不这样做，可能会导致严重的性能损失。本文提出了一种系统的方法，利用多面体模型来分析所有的运营商的DL模型在一起，以减少内存访问的数量。实验表明，我们的方法可以大大减少常见神经网络模型所需的内存访问对自主开发的AWS机器学习推理芯片Inferentia的影响，该芯片可通过Amazon EC2 Inf 1实例获得。
英文摘要：
Deep learning (DL) workloads are moving towards accelerators for faster processing and lower cost. Modern DL accelerators are good at handling the large-scale multiply-accumulate operations that dominate DL workloads; however, it is challenging to make full use of the compute power of an accelerator since the data must be properly staged in a software-managed scratchpad memory. Failing to do so can result in significant performance loss. This paper proposes a systematic approach which leverages the polyhedral model to analyze all operators of a DL model together to minimize the number of memory accesses. Experiments show that our approach can substantially reduce the impact of memory accesses required by common neural-network models on a homegrown AWS machine-learning inference chip named Inferentia, which is available through Amazon EC2 Inf1 instances.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
