arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.00636cs.LGcs.CV
收藏

将图解释应用于算子融合

Applying Graph Explanation to Operator Fusion

Huawei Technologies(华为技术有限公司);Huawei Kirin Solution(华为麒麟解决方案);University of Alberta(阿尔伯塔大学)

Keith G. Mills, Muhammad Fetrat Qharabagh, Weichen Qiu, Fred X. Han, Mohammad Salameh, Wei Lu, Shangling Jui, Di Niu

AI总结:

本文提出将图解释技术引入DNN层融合,通过识别无效融合组中的关键操作并以贪心树算法递归拆分来最小化DRAM访问,在EfficientNet-B3上实现超过20%的DRAM访问减少。

摘要:

层融合技术对于提高深度神经网络(DNN)部署时的推理效率至关重要。融合旨在通过减少加速器片上缓冲区和DRAM之间的数据事务来降低推理成本。这是通过将卷积和激活等多个操作分组执行为单一执行单元——融合组来实现的。然而,片上缓冲区容量限制了融合组的大小,在整个DNN上优化融合需要将其划分为多个融合组。寻找最优融合组是一个复杂问题,其中无效解的存在阻碍了传统搜索算法,并要求采用鲁棒的方法。在本文中,我们将可解释人工智能,特别是图解释技术(GET),引入到层融合中。给定一个无效融合组,我们识别出对组无效性最负责的操作,然后利用这一知识通过基于贪心树的算法递归地拆分原始融合组,以最小化DRAM访问。我们将我们的方案与常见算法配对,并在两种类型的层融合上优化DNN:行缓冲深度优先(LBDF)和分支需求减少(BRR)。实验证明了我们的方案在多个流行和经典的卷积神经网络(如ResNets和MobileNets)上的有效性。我们的方案在EfficientNet-B3上实现了超过20%的DRAM访问减少。

英文摘要:

Layer fusion techniques are critical to improving the inference efficiency of deep neural networks (DNN) for deployment. Fusion aims to lower inference costs by reducing data transactions between an accelerator's on-chip buffer and DRAM. This is accomplished by grouped execution of multiple operations like convolution and activations together into single execution units - fusion groups. However, on-chip buffer capacity limits fusion group size and optimizing fusion on whole DNNs requires partitioning into multiple fusion groups. Finding the optimal groups is a complex problem where the presence of invalid solutions hampers traditional search algorithms and demands robust approaches. In this paper we incorporate Explainable AI, specifically Graph Explanation Techniques (GET), into layer fusion. Given an invalid fusion group, we identify the operations most responsible for group invalidity, then use this knowledge to recursively split the original fusion group via a greedy tree-based algorithm to minimize DRAM access. We pair our scheme with common algorithms and optimize DNNs on two types of layer fusion: Line-Buffer Depth First (LBDF) and Branch Requirement Reduction (BRR). Experiments demonstrate the efficacy of our scheme on several popular and classical convolutional neural networks like ResNets and MobileNets. Our scheme achieves over 20% DRAM Access reduction on EfficientNet-B3.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑