GFormer：在Gaudi处理器上利用优化Transformer加速大语言模型
GFormer: Accelerating Large Language Models with Optimized Transformers on Gaudi Processors
AI总结：
本文提出GFormer方法，通过融合稀疏与线性注意力并设计窗口化自注意力和高效外积内核，在Gaudi处理器上优化大语言模型推理，显著提升效率与性能并超越主流GPU。
摘要：
异构硬件（如Gaudi处理器）已被开发用于增强计算能力，尤其是面向生成式AI任务中基于Transformer的大语言模型（LLMs）的矩阵运算。然而，我们的分析表明，Transformer在这类新兴硬件上并未得到充分优化，主要原因是非矩阵计算内核（如Softmax）以及异构资源利用方面的优化不足，尤其是在处理长序列时。为解决这些问题，我们提出一种融合稀疏注意力和线性注意力机制的综合方法（称为GFormer）。GFormer旨在最大化Gaudi处理器的矩阵乘法引擎（MME）和张量处理核心（TPC）的计算能力，同时不损害模型质量。GFormer包含一个窗口化自注意力内核和一个用于因果线性注意力的高效外积内核，旨在优化Gaudi处理器上的LLM推理。评估结果表明，GFormer在Gaudi处理器上显著提升了各种任务的效率和模型性能，并优于最先进的GPU。
英文摘要：
Heterogeneous hardware like Gaudi processor has been developed to enhance computations, especially matrix operations for Transformer-based large language models (LLMs) for generative AI tasks. However, our analysis indicates that Transformers are not fully optimized on such emerging hardware, primarily due to inadequate optimizations in non-matrix computational kernels like Softmax and in heterogeneous resource utilization, particularly when processing long sequences. To address these issues, we propose an integrated approach (called GFormer) that merges sparse and linear attention mechanisms. GFormer aims to maximize the computational capabilities of the Gaudi processor's Matrix Multiplication Engine (MME) and Tensor Processing Cores (TPC) without compromising model quality. GFormer includes a windowed self-attention kernel and an efficient outer product kernel for causal linear attention, aiming to optimize LLM inference on Gaudi processors. Evaluation shows that GFormer significantly improves efficiency and model performance across various tasks on the Gaudi processor and outperforms state-of-the-art GPUs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
