arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2004.05333cs.LGcs.PF
收藏

用于神经加速的位并行向量组合

Bit-Parallel Vector Composability for Neural Acceleration

University of California, San Diego(加州大学圣迭戈分校);Bigstream, Inc.(Bigstream公司);Google Inc.(谷歌公司);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)

Soroush Ghodrati, Hardik Sharma, Cliff Young, Nam Sung Kim, Hadi Esmaeilzadeh

摘要:

传统的神经加速器依赖于隔离的自给自足的功能单元,这些功能单元执行原子操作,同时通过操作数传递-聚合逻辑来传达结果。每个单元都原子地处理其操作数的所有位,并隔离地产生结果的所有位。本文探索了一种不同的设计风格,其中每个单元只负责一部分位级操作,以交错并结合位级并行的好处与深度神经网络中丰富的数据级并行。这些单元的动态集合在运行时协作以共同生成结果的位。这种协作需要在位之间提取新的分组,这仅在操作数和操作是可向量化的情况下是可能的。丰富的数据级并行性和大多数重复的执行模式为定义和利用位并行向量可组合性的新维度提供了独特的机会。这种设计将位并行性分散在数据级并行性中,并动态地将两者交织在一起。因此,我们的神经加速器的构建块是一个可组合向量单元,它是一个窄位宽向量引擎的集合,这些引擎以位粒度动态组合或分解。使用六种不同的CNN和LSTM深度网络,我们在四个设计点上评估了这种设计风格:有和没有算法位宽异构性,以及有和没有高带宽片外存储器。在这四个设计点上,位并行矢量可组合性带来了(1.4倍至3.5倍)的加速和(1.1倍至2.7倍)的能耗降低。我们还将我们的设计风格与也支持INT-4执行的Nvidia RTX 2080 TI GPU进行了全面比较。其优势范围为每瓦性能提高28.0倍至33.7倍。

英文摘要:

Conventional neural accelerators rely on isolated self-sufficient functional units that perform an atomic operation while communicating the results through an operand delivery-aggregation logic. Each single unit processes all the bits of their operands atomically and produce all the bits of the results in isolation. This paper explores a different design style, where each unit is only responsible for a slice of the bit-level operations to interleave and combine the benefits of bit-level parallelism with the abundant data-level parallelism in deep neural networks. A dynamic collection of these units cooperate at runtime to generate bits of the results, collectively. Such cooperation requires extracting new grouping between the bits, which is only possible if the operands and operations are vectorizable. The abundance of Data Level Parallelism and mostly repeated execution patterns, provides a unique opportunity to define and leverage this new dimension of Bit-Parallel Vector Composability. This design intersperses bit parallelism within data-level parallelism and dynamically interweaves the two together. As such, the building block of our neural accelerator is a Composable Vector Unit that is a collection of Narrower-Bitwidth Vector Engines, which are dynamically composed or decomposed at the bit granularity. Using six diverse CNN and LSTM deep networks, we evaluate this design style across four design points: with and without algorithmic bitwidth heterogeneity and with and without availability of a high-bandwidth off-chip memory. Across these four design points, Bit-Parallel Vector Composability brings (1.4x to 3.5x) speedup and (1.1x to 2.7x) energy reduction. We also comprehensively compare our design style to the Nvidia RTX 2080 TI GPU, which also supports INT-4 execution. The benefits range between 28.0x and 33.7x improvement in Performance-per-Watt.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑