IterL2Norm：快速迭代式L2归一化
IterL2Norm: Fast Iterative L2-Normalization
AI总结：
针对Transformer大模型层归一化需减少跨芯片数据传输的需求，提出迭代L2归一化方法IterL2Norm，5步内收敛且精度多优于快速逆平方根算法，流片实现后可低延迟处理常用维度向量。
摘要：
基于Transformer的大语言模型属于内存绑定型模型，其运行依赖大量复用率极低的数据，因此主机与加速器之间的数据传输往往决定了总运行时长。层归一化是Transformer模型中的关键运算负载之一，紧随每个多头注意力模块和前馈网络模块之后。为减少数据传输，层归一化需要与矩阵乘法引擎在同一芯片上执行。为此，我们提出了一种面向一维输入的迭代L2归一化方法IterL2Norm，该方法可在5次迭代内快速收敛至稳态解且精度较高，在OPT模型所用的各嵌入长度下，FP32精度的9种测试场景中有6种优于快速逆平方根算法，BFloat16精度的9种场景中有5种更优。IterL2Norm宏单元采用32/28nm CMOS工艺实现，可对维度64≤d≤1024的d维向量做归一化，在100MHz/1.05V条件下延迟为116-227个周期。
英文摘要：
Transformer-based large language models are a memory-bound model whose operation is based on a large amount of data that are marginally reused. Thus, the data movement between a host and accelerator likely dictates the total wall-clock time. Layer normalization is one of the key workloads in the transformer model, following each of multi-head attention and feed-forward network blocks. To reduce data movement, layer normalization needs to be performed on the same chip as the matrix-matrix multiplication engine. To this end, we introduce an iterative L2-normalization method for 1D input (IterL2Norm), ensuring fast convergence to the steady-state solution within five iteration steps and high precision, outperforming the fast inverse square root algorithm in six out of nine cases for FP32 and five out of nine for BFloat16 across the embedding lengths used in the OPT models. Implemented in 32/28nm CMOS, the IterL2Norm macro normalizes $d$-dimensional vectors, where $64 \leq d \leq 1024$, with a latency of 116-227 cycles at 100MHz/1.05V.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
