ESACT: 一种端到端的稀疏加速器，通过局部相似性用于计算密集型Transformer
ESACT: An End-to-End Sparse Accelerator for Compute-Intensive Transformers via Local Similarity
AI总结：
ESACT通过局部相似性预测机制，实现端到端稀疏加速，减少计算量达52.03%，提升能效3.29 TOPS/W。
摘要：
Transformer，由QKV生成、注意力计算和FFN组成，由于其卓越的性能，已成为各种领域的主导模型。然而，其高计算成本阻碍了高效的硬件部署。稀疏性提供了一个有前景的解决方案，但大多数现有加速器只利用注意力中的行内稀疏性，而很少考虑行间稀疏性。利用行间稀疏性的方法通常依赖于成本高昂的全局相似性估计，这会削弱稀疏性的加速效益，并且通常只将稀疏性应用于一个或两个Transformer组件。通过仔细分析注意力分布和计算流程，我们发现局部相似性允许端到端的稀疏加速，具有较低的计算开销。受这一观察的启发，我们提出了ESACT，一种用于计算密集型Transformer的端到端稀疏加速器。ESACT的核心是基于局部相似性的稀疏性预测（SPLS）机制，该机制利用HLog量化在QK生成之前准确预测局部注意力稀疏性先验，从而在所有Transformer组件上实现高效的稀疏性。为了支持高效的硬件实现，我们引入了三种架构创新。在26个基准测试中的实验结果表明，SPLS将总计算量减少了52.03%，同时精度损失小于1%。ESACT实现了端到端的能效3.29 TOPS/W，并在SOTA注意力加速器SpAtten和Sanger上分别提高了注意力层面的能效2.95倍和2.26倍。
英文摘要：
Transformers, composed of QKV generation, attention computation, and FFNs, have become the dominant model across various domains due to their outstanding performance. However, their high computational cost hinders efficient hardware deployment. Sparsity offers a promising solution, yet most existing accelerators exploit only intra-row sparsity in attention, while few consider inter-row sparsity. Approaches leveraging inter-row sparsity often rely on costly global similarity estimation, which diminishes the acceleration benefits of sparsity, and typically apply sparsity to only one or two transformer components. Through careful analysis of the attention distribution and computation flow, we observe that local similarity allows end-to-end sparse acceleration with lower computational overhead. Motivated by this observation, we propose ESACT, an end-to-end sparse accelerator for compute-intensive Transformers. ESACT centers on the Sparsity Prediction with Local Similarity (SPLS) mechanism, which leverages HLog quantization to accurately predict local attention sparsity prior to QK generation, achieving efficient sparsity across all transformer components. To support efficient hardware realization, we introduce three architectural innovations. Experimental results on 26 benchmarks demonstrate that SPLS reduces total computation by 52.03% with less than 1% accuracy loss. ESACT achieves an end-to-end energy efficiency of 3.29 TOPS/W, and improves attention-level energy efficiency by 2.95x and 2.26x over SOTA attention accelerators SpAtten and Sanger, respectively.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
