arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2304.14463cs.LGcs.AI
收藏

Moccasin:面向神经网络的高效张量重物化方法

Moccasin: Efficient Tensor Rematerialization for Neural Networks

Qualcomm AI Research(高通人工智能研究院);University of Southern California(南加州大学)

Burak Bartan, Haoming Li, Harris Teague, Christopher Lott, Bistra Dilkina

AI总结:

本文提出 Moccasin 约束规划方法,将张量重物化问题变量数从 O(n^2) 降至 O(n),在内存预算下最小化计算图执行时间,实验表明其在大规模图上比现有方法快最多一个数量级。

摘要:

神经网络在边缘计算设备上的部署与训练面临诸多挑战。边缘设备内存有限,往往是部署大型神经网络模型时遇到的最大限制因素之一。张量重物化(tensor rematerialization)或重计算(recompute)是解决神经网络训练和推理中高内存需求的一种方法。本文研究在内存预算约束下最小化计算图执行时间的问题。具体而言,我们提出了一种新的约束规划(constraint programming)形式化方法,称为 Moccasin,仅使用 O(n) 个整数变量,其中 n 是计算图中的节点数。与近期文献中提出的使用 O(n^2) 个布尔变量的形式化方法相比,这是一个显著改进。我们给出的数值研究表明,我们的方法比近期工作快最多一个数量级,尤其是在大规模图上。

英文摘要:

The deployment and training of neural networks on edge computing devices pose many challenges. The low memory nature of edge devices is often one of the biggest limiting factors encountered in the deployment of large neural network models. Tensor rematerialization or recompute is a way to address high memory requirements for neural network training and inference. In this paper we consider the problem of execution time minimization of compute graphs subject to a memory budget. In particular, we develop a new constraint programming formulation called \textsc{Moccasin} with only $O(n)$ integer variables, where $n$ is the number of nodes in the compute graph. This is a significant improvement over the works in the recent literature that propose formulations with $O(n^2)$ Boolean variables. We present numerical studies that show that our approach is up to an order of magnitude faster than recent work especially for large-scale graphs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑