arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1804.10001cs.DCcs.AIcs.LG
收藏

深度神经网络的轮廓引导存储优化

Profile-guided memory optimization for deep neural networks

National Institute of Informatics(国立信息学研究所);IBM Research – Tokyo(IBM东京研究院)

Taro Sekiyama, Takashi Imamichi, Haruki Imai, Rudy Raymond

摘要:

近年来,深度神经网络(DNN)变得越来越广泛和深入,以在许多AI应用中实现更好的性能。然而,这样的DNN需要大量的存储器来存储权重和中间结果(例如,激活、特征图等)in propagation传播.这一要求使得DNN难以在内存有限、难以扩展的设备上运行,降低了运行时间性能,并限制了网络模型的设计。我们通过开发一种新的配置文件引导的内存优化来解决这一挑战,以在DNN中的传播过程中高效快速地分配内存块。该优化方法基于二维矩形布局问题,采用了一种简单快速的启发式算法。与著名的神经网络模型的实验,我们证实,我们的方法不仅减少了内存消耗高达$49.5\%$,但也加快了训练和推理的四个因素由于内存分配的速度和使用更大的小批量大小的能力。

英文摘要:

Recent years have seen deep neural networks (DNNs) becoming wider and deeper to achieve better performance in many applications of AI. Such DNNs however require huge amounts of memory to store weights and intermediate results (e.g., activations, feature maps, etc.) in propagation. This requirement makes it difficult to run the DNNs on devices with limited, hard-to-extend memory, degrades the running time performance, and restricts the design of network models. We address this challenge by developing a novel profile-guided memory optimization to efficiently and quickly allocate memory blocks during the propagation in DNNs. The optimization utilizes a simple and fast heuristic algorithm based on the two-dimensional rectangle packing problem. Experimenting with well-known neural network models, we confirm that our method not only reduces the memory consumption by up to $49.5\%$ but also accelerates training and inference by up to a factor of four thanks to the rapidity of the memory allocation and the ability to use larger mini-batch sizes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑