TrIM:面向卷积神经网络的三角输入移动脉动阵列:数据流与分析建模
TrIM, Triangular Input Movement Systolic Array for Convolutional Neural Networks: Dataflow and Analytical Modelling
AI总结:
针对卷积神经网络中脉动阵列存在的数据冗余与内存开销问题,提出基于三角输入移动的新型数据流TrIM,在减少内存访问、提升吞吐量与降低寄存器需求上表现更优。
摘要:
为了适配前沿AI模型持续增长的计算复杂度与数据强度,研究人员正在提出新的计算范式。这些范式旨在通过缓解冯·诺依曼瓶颈实现高能效,该瓶颈与处理核心和内存之间数据移动的能耗成本相关。卷积神经网络(Convolutional Neural Networks, CNNs)由于需要处理海量数据,极易受该瓶颈影响。脉动阵列(Systolic Arrays, SAs)凭借处理单元(Processing Elements, PEs)的高数据利用率,是缓解数据传输成本的极具潜力的架构。这些PE基于特定数据流(如权重静止、行静止)在本地持续交换和处理数据,进而减少对主存的内存访问次数。在脉动阵列中,卷积要么被当作矩阵乘法处理,要么利用滑动窗口的光栅顺序扫描来实现。然而,数据冗余是影响面积、功耗与能耗的核心问题。本文提出TrIM:一种基于三角输入移动、适配CNN计算的新型脉动阵列数据流。TrIM最大化本地输入利用率,最小化权重数据移动,并解决了数据冗余问题。此外,TrIM不会产生行静止数据流带来的显著片上内存开销。与前沿脉动阵列数据流相比,TrIM提供的高数据利用率可保证内存访问量减少约10倍。此外,考虑到PE持续重叠乘法与累加操作,TrIM除了所需寄存器数量有限(比行静止少最多15.6倍),还实现了高吞吐量(比行静止高最多81.8%)。
英文摘要:
In order to follow the ever-growing computational complexity and data intensity of state-of-the-art AI models, new computing paradigms are being proposed. These paradigms aim at achieving high energy efficiency by mitigating the Von Neumann bottleneck that relates to the energy cost of moving data between the processing cores and the memory. Convolutional Neural Networks (CNNs) are susceptible to this bottleneck, given the massive data they have to manage. Systolic arrays (SAs) are promising architectures to mitigate data transmission cost, thanks to high data utilization of Processing Elements (PEs). These PEs continuously exchange and process data locally based on specific dataflows (such as weight stationary and row stationary), in turn reducing the number of memory accesses to the main memory. In SAs, convolutions are managed either as matrix multiplications or exploiting the raster-order scan of sliding windows. However, data redundancy is a primary concern affecting area, power, and energy. In this paper, we propose TrIM: a novel dataflow for SAs based on a Triangular Input Movement and compatible with CNN computing. TrIM maximizes the local input utilization, minimizes the weight data movement, and solves the data redundancy problem. Furthermore, TrIM does not incur the significant on-chip memory penalty introduced by the row stationary dataflow. When compared to state-of-the-art SA dataflows, the high data utilization offered by TrIM guarantees ~10X less memory access. Furthermore, considering that PEs continuously overlap multiplications and accumulations, TrIM achieves high throughput (up to 81.8% higher than row stationary), other than requiring a limited number of registers (up to 15.6X fewer registers than row stationary).
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。