HeTraX：面向Transformer加速的节能三维异构众核架构
HeTraX: Energy Efficient 3D Heterogeneous Manycore Architecture for Transformer Acceleration
AI总结：
本文提出三维异构众核架构 HeTraX，通过硬件资源与 Transformer 计算内核对齐，在保证热可行性的同时实现最高 5.6 倍加速和 14.5 倍 EDP 改善。
摘要：
Transformer 已经彻底改变了深度学习和生成建模，使自然语言处理任务及其他领域取得了前所未有的进步。然而，由于 Transformer 架构中涉及的计算内核种类繁多，设计用于执行 Transformer 模型的硬件加速器极具挑战性。现有加速器要么不足以加速端到端 Transformer 模型，要么存在显著的热限制。在本文中，我们提出了一种三维异构架构的设计，称为 HeTraX，专门针对端到端 Transformer 模型的加速进行了优化。HeTraX 采用与 Transformer 计算内核相匹配的硬件资源，并同时优化性能和能耗。实验结果表明，HeTraX 在速度上比现有最先进技术最高提升 5.6 倍，并将 EDP 改善 14.5 倍，同时确保热可行性。
英文摘要：
Transformers have revolutionized deep learning and generative modeling to enable unprecedented advancements in natural language processing tasks and beyond. However, designing hardware accelerators for executing transformer models is challenging due to the wide variety of computing kernels involved in the transformer architecture. Existing accelerators are either inadequate to accelerate end-to-end transformer models or suffer notable thermal limitations. In this paper, we propose the design of a three-dimensional heterogeneous architecture referred to as HeTraX specifically optimized to accelerate end-to-end transformer models. HeTraX employs hardware resources aligned with the computational kernels of transformers and optimizes both performance and energy. Experimental results show that HeTraX outperforms existing state-of-the-art by up to 5.6x in speedup and improves EDP by 14.5x while ensuring thermally feasibility.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
