SISA:一种可扩展的流体阵列用于GEMM加速
SISA: A Scale-In Systolic Array for GEMM Acceleration
AI总结:
针对LLM中非均匀矩阵带来的Systolic Array资源浪费问题,SISA通过将传统正方形阵列划分为水平矩形片,实现小/倾斜矩阵的高效执行,同时保留大GEMM的全阵操作,从而在代表LLM上实现8.52倍的速度提升和93%的EDP降低。
摘要:
针对LLM中非均匀矩阵带来的Systolic Array资源浪费问题,SISA通过将传统正方形阵列划分为水平矩形片,实现小/倾斜矩阵的高效执行,同时保留大GEMM的全阵操作,从而在代表LLM上实现8.52倍的速度提升和93%的EDP降低。
英文摘要:
The currently dominant AI/ML workloads, such as Large Language Models (LLMs), rely on the efficient execution of General Matrix-Matrix Multiplication (GEMM) operations. Thus, most systems are equipped with dedicated matrix hardware accelerators based on square Systolic Arrays (SAs) of Processing Elements (PEs). While this organization was effective for traditional Deep Neural Networks (DNNs), LLMs introduce input-dependent and highly skewed matrices, leading to underutilized SA resources. To address this challenge, we propose SISA (Scale-In Systolic Array), a novel SA architecture that partitions the traditional square array into horizontal rectangular slabs. With minimal overhead, SISA exposes parallelism through independently scheduled slabs for efficient execution of small or skewed matrix shapes, while retaining full-array operation for large GEMMs. SISA achieves up to 8.52x speedup and 93% energy-delay-product (EDP) reduction for representative LLMs compared to a state-of-the-art monolithic SA with the same number of PEs.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。