Allspark：面向内存处理系统中视觉Transformer的工作负载编排
Allspark: Workload Orchestration for Visual Transformers on Processing In-Memory Systems
AI总结：
Allspark提出面向PIM系统的视觉Transformer工作负载编排方案，通过细粒度划分、ILP调度和两阶段放置，实现1.2-24倍推理加速及20-55倍能耗节省。
摘要：
Transformer的出现彻底改变了计算机视觉领域，为卷积神经网络（CNN）提供了一种强大的替代方案，尤其是其局部注意力机制擅长捕捉输入中的局部结构并实现最先进的性能。内存处理（PIM）架构提供了大规模并行性、低数据移动成本和可扩展的内存带宽，使其成为加速具有内存密集型操作的Transformer的有前景的解决方案。然而，关键问题在于如何将整个模型高效地部署到资源受限的PIM系统上，同时基于局部注意力机制将每个Transformer块并行化为可能具有多个计算分支的结构。我们提出了Allspark，专注于PIM系统上视觉Transformer的工作负载编排，旨在最小化推理延迟。首先，为了充分利用PIM的大规模并行性，Allspark对计算分支采用细粒度划分方案，并设计了系统化的布局和交错数据流，以最大化数据局部性并减少数据移动。其次，Allspark将完整模型在资源受限的分布式PIM系统上的调度问题表述为整数线性规划（ILP）问题。第三，由于局部-全局数据交互表现出复杂但规则的依赖关系，Allspark提供了一种两阶段放置方法，将计算分支在PIM系统上具有挑战性的放置问题简化为结构化布局和基于贪心的绑定，以最小化NoC通信成本。在基于3D堆叠DRAM的PIM系统上进行的大量实验表明，与基线相比，Allspark为各种视觉Transformer带来了1.2倍至24.0倍的推理加速。与Nvidia V100 GPU相比，Allspark增强的PIM系统平均加速2.3倍，并节省20倍至55倍的能耗。
英文摘要：
The advent of Transformers has revolutionized computer vision, offering a powerful alternative to convolutional neural networks (CNNs), especially with the local attention mechanism that excels at capturing local structures within the input and achieve state-of-the-art performance. Processing in-memory (PIM) architecture offers extensive parallelism, low data movement costs, and scalable memory bandwidth, making it a promising solution to accelerate Transformer with memory-intensive operations. However, the crucial issue lies in efficiently deploying an entire model onto resource-limited PIM system while parallelizing each transformer block with potentially many computational branches based on local-attention mechanisms. We present Allspark, which focuses on workload orchestration for visual Transformers on PIM systems, aiming at minimizing inference latency. Firstly, to fully utilize the massive parallelism of PIM, Allspark employs a fine-grained partitioning scheme for computational branches, and formats a systematic layout and interleaved dataflow with maximized data locality and reduced data movement. Secondly, Allspark formulates the scheduling of the complete model on a resource-limited distributed PIM system as an integer linear programming (ILP) problem. Thirdly, as local-global data interactions exhibit complex yet regular dependencies, Allspark provides a two-stage placement method, which simplifies the challenging placement of computational branches on the PIM system into the structured layout and greedy-based binding, to minimize NoC communication costs. Extensive experiments on 3D-stacked DRAM-based PIM systems show that Allspark brings 1.2x-24.0x inference speedup for various visual Transformers over baselines. Compared to Nvidia V100 GPU, Allspark-enriched PIM system yields average speedups of 2.3x and energy savings of 20x-55x.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
