MARCA:采用可重构架构的Mamba加速器
MARCA: Mamba Accelerator with ReConfigurable Architecture
AI总结:
本研究提出可重构架构的Mamba加速器MARCA,通过三类创新设计,在不同规模Mamba模型上实现了远超CPU和GPU的加速比与能效提升。
摘要:
我们提出了一种采用可重构架构的Mamba加速器MARCA。本文提出了三种创新方法:(1)面向线性运算和逐元素运算的约简备选PE阵列架构。针对线性运算,连接PE阵列的约简树启用并执行约简操作;针对逐元素运算,约简树禁用且输出直接旁路。(2)基于可重构PE的可复用非线性函数单元。我们通过快速有偏指数算法将指数函数分解为逐元素运算和移位运算,通过分段近似算法将激活函数(SiLU)分解为范围检测和逐元素运算,从而复用可重构PE执行非线性函数,且精度损失可忽略不计。(3)运算内与运算间缓存管理策略:提出运算内缓存管理策略,最大化线性运算在运算内部的输入数据共享;针对运算间的逐元素运算,采用运算间策略。我们在不同规模的Mamba模型家族上开展了大量实验。与Intel Xeon 8358P CPU和NVIDIA Tesla A100 GPU的实现相比,MARCA分别实现了最高463.22倍/11.66倍的加速比,以及最高9761.42倍/242.52倍的能效提升。
英文摘要:
We propose a Mamba accelerator with reconfigurable architecture, MARCA.We propose three novel approaches in this paper. (1) Reduction alternative PE array architecture for both linear and element-wise operations. For linear operations, the reduction tree connected to PE arrays is enabled and executes the reduction operation. For element-wise operations, the reduction tree is disabled and the output bypasses. (2) Reusable nonlinear function unit based on the reconfigurable PE. We decompose the exponential function into element-wise operations and a shift operation by a fast biased exponential algorithm, and the activation function (SiLU) into a range detection and element-wise operations by a piecewise approximation algorithm. Thus, the reconfigurable PEs are reused to execute nonlinear functions with negligible accuracy loss.(3) Intra-operation and inter-operation buffer management strategy. We propose intra-operation buffer management strategy to maximize input data sharing for linear operations within operations, and inter-operation strategy for element-wise operations between operations. We conduct extensive experiments on Mamba model families with different sizes.MARCA achieves up to 463.22$\times$/11.66$\times$ speedup and up to 9761.42$\times$/242.52$\times$ energy efficiency compared to Intel Xeon 8358P CPU and NVIDIA Tesla A100 GPU implementations, respectively.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。