arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2107.02547cs.AR
收藏

可变形卷积网络的节能加速器设计

Energy-Efficient Accelerator Design for Deformable Convolution Networks

Dawen Xu, Cheng Chu, Cheng Liu, Ying Wang, Huawei Li, Xiaowei Li, Kwang-Ting Cheng

摘要:

可变形卷积网络(DCN)被提出来解决具有几何或光度变化的图像识别问题,通常涉及在输入特征的任意位置上进行卷积的可变形卷积。位置随不同的输入而变化,并引起相当大的动态和不规则的内存访问,这是经典的神经网络加速器(NNA)无法处理的。此外,在DCN中获得变形特征所需的双线性插值(BLI)操作也不能直接部署在现有的NNA上。虽然与经典NNA一起安置的通用处理器(GPP)可以处理可变形卷积,但是由于缺乏并行计算能力,GPP上的处理可能非常慢。为了解决这个问题,我们在现有的NNA上开发了一个DCN加速器,以支持标准卷积和变形卷积。具体来说,对于DCN中的动态和不规则访问,我们将输入和输出特征划分为瓦片,并构建瓦片依赖性表(TDT)以在运行时跟踪不规则瓦片依赖性。利用TDT,我们进一步开发了一个片上瓦片调度器来有效地处理动态和不规则的访问。此外,我们提出了一种新的映射策略,使并行BLI处理NNA和应用层融合技术,更节能的DCN处理。实验结果表明,该加速器的性能和能量效率比ARM、ARM+TPU和GPU等典型计算架构提高了几个数量级,而传统NNA的芯片面积损失仅为6.6%.

英文摘要:

Deformable convolution networks (DCNs) proposed to address the image recognition with geometric or photometric variations typically involve deformable convolution that convolves on arbitrary locations of input features. The locations change with different inputs and induce considerable dynamic and irregular memory accesses which cannot be handled by classic neural network accelerators (NNAs). Moreover, bilinear interpolation (BLI) operation that is required to obtain deformed features in DCNs also cannot be deployed on existing NNAs directly. Although a general purposed processor (GPP) seated along with classic NNAs can process the deformable convolution, the processing on GPP can be extremely slow due to the lack of parallel computing capability. To address the problem, we develop a DCN accelerator on existing NNAs to support both the standard convolution and deformable convolution. Specifically, for the dynamic and irregular accesses in DCNs, we have both the input and output features divided into tiles and build a tile dependency table (TDT) to track the irregular tile dependency at runtime. With the TDT, we further develop an on-chip tile scheduler to handle the dynamic and irregular accesses efficiently. In addition, we propose a novel mapping strategy to enable parallel BLI processing on NNAs and apply layer fusion techniques for more energy-efficient DCN processing. According to our experiments, the proposed accelerator achieves orders of magnitude higher performance and energy efficiency compared to the typical computing architectures including ARM, ARM+TPU, and GPU with 6.6\% chip area penalty to a classic NNA.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑