CNN加速中的计算单元划分
Partitioning Compute Units in CNN Acceleration for Statistical Memory Traffic Shaping
摘要:
CNN的设计复杂性一直在稳步增加,以提高准确性。为了应对这种复杂CNN所需的大量计算,最新的解决方案利用在可用维度上的图像分块和多个输入图像的分块来提高存储层次结构中的数据重用。虽然已经有大量的工作,最大限度地提高数据重用,只有少数研究集中在有限的带宽造成的内存瓶颈。在CNN加速中很容易出现带宽瓶颈,因为CNN层具有不同的大小,具有不同的计算需求,并且通常在每个CNN层上执行卷积以实现理想的数据重用。在这种情况下,与层的计算需求相比,层的数据传输需求可能相对较低或较高,因此可能引起存储器访问的时间波动,最终导致带宽问题。在本文中,我们首先表明,存储器访问与计算比率存在高度波动,这取决于CNN层和计算单元(核心)处理的层中的功能,其中单元紧密同步以最大限度地提高数据重用。然后,我们提出了一个分区的计算单元,其中每个分区内的核心同步处理一批输入数据,以最大限度地提高数据重用,但不同的分区异步运行的策略。由于分区保持异步并且通常在任何给定时刻处理不同的CNN层,因此分区的存储器访问流量大小变得在统计上混洗。因此,计算单元的分区和它们的异步使用使得总存储器访问流量大小随时间平滑。我们称之为平滑统计内存流量整形,我们表明,在运行ResNet-50时,它可以在商用64核处理器上带来8.0%的性能增益。
英文摘要:
The design complexity of CNNs has been steadily increasing to improve accuracy. To cope with the massive amount of computation needed for such complex CNNs, the latest solutions utilize blocking of an image over the available dimensions and batching of multiple input images to improve data reuse in the memory hierarchy. While there has been numerous works on maximizing data reuse, only a few studies have focused on the memory bottleneck caused by limited bandwidth. Bandwidth bottleneck can easily occur in CNN acceleration as CNN layers have different sizes with varying computation needs and as batching is typically performed over each CNN layer for an ideal data reuse. In this case, the data transfer demand for a layer can be relatively low or high compared to the computation requirement of the layer, and hence temporal fluctuations in memory access can be induced eventually causing bandwidth problems. In this paper, we first show that there exists a high degree of fluctuation in memory access to computation ratio depending on CNN layers and functions in the layer being processed by the compute units (cores), where the units are tightly synchronized to maximize data reuse. Then we propose a strategy of partitioning the compute units where the cores within each partition process a batch of input data synchronously to maximize data reuse but different partitions run asynchronously. As the partitions stay asynchronous and typically process different CNN layers at any given moment, the memory access traffic sizes of the partitions become statistically shuffled. Thus, the partitioning of compute units and asynchronous use of them make the total memory access traffic size be smoothened over time. We call this smoothing statistical memory traffic shaping, and we show that it can lead to 8.0 percent of performance gain on a commercial 64-core processor when running ResNet-50.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。