arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.14393cs.AR
收藏

Scope:一种可扩展的多芯片模块NN加速器合并流水线框架

Scope: A Scalable Merged Pipeline Framework for Multi-Chip-Module NN Accelerators

Tsinghua University(清华大学);Beijing National Research Center for Information Science and Technology(北京信息科学与技术国家研究中心);Capital Normal University(首都师范大学)

Zongle Huang, Hongyang Jia, Kaiwei Zou, Yongpan Liu

AI总结:

Scope提出了一种合并流水线框架,通过整合多层维度优化多芯片模块NN加速器的吞吐量和可扩展性,实验显示其在ResNet-152推理中实现了1.73倍的吞吐量提升。

摘要:

神经网络(NN)加速器采用多芯片模块(MCM)架构可以实现大规模计算能力的集成;然而,它们面临计算资源利用率低和芯片间通信开销的问题。传统的并行化方案,如层内并行和层间流水线,无法突破这两个挑战,限制了MCM架构的可扩展性。我们发现现有工作通常将层单独部署,而非联合考虑。这种未被充分利用的维度导致系统计算与通信之间的权衡,从而阻碍了最优利用,尤其是随着硬件/软件规模的扩大。为了解决这一限制,我们提出了Scope,一种合并流水线框架,将这一被忽视的多层维度纳入其中,从而通过放松计算、通信和内存成本之间的权衡,实现更高的吞吐量和可扩展性。然而,这一新维度增加了设计空间探索(DSE)的复杂性。为此,我们开发了一系列搜索算法,实现了指数到线性的复杂度降低,同时识别出排名在前0.05%的解决方案。实验表明,Scope在ResNet-152推理中相比最先进的方法实现了高达1.73倍的吞吐量提升,同时保持了相似的能量消耗。

英文摘要:

Neural network (NN) accelerators with multi-chip-module (MCM) architectures enable integration of massive computation capability; however, they face challenges of computing resource underutilization and off-chip communication overheads. Traditional parallelization schemes for NN inference on MCM architectures, such as intra-layer parallelism and inter-layer pipelining, show incompetency in breaking through both challenges, limiting the scalability of MCM architectures. We observed that existing works typically deploy layers separately rather than considering them jointly. This underexploited dimension leads to compromises between system computation and communication, thus hindering optimal utilization, especially as hardware/software scale. To address this limitation, we propose Scope, a merged pipeline framework incorporating this overlooked multi-layer dimension, thereby achieving improved throughput and scalability by relaxing tradeoffs between computation, communication and memory costs. This new dimension, however, adds to the complexity of design space exploration (DSE). To tackle this, we develop a series of search algorithms that achieves exponential-to-linear complexity reduction, while identifying solutions that rank in the top 0.05% of performance. Experiments show that Scope achieves up to 1.73x throughput improvement while maintaining similar energy consumption for ResNet-152 inference compared to state-of-the-art approaches.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑