通过动态块级回退实现精确的 INT8 训练
Accurate INT8 Training Through Dynamic Block-Level Fallback
AI总结：
针对含 GLU 的现代 Transformer 因激活异常值导致 INT8 训练困难的问题，提出 Fallback Quantization，对异常值激活块动态回退至 16 位混合精度 GEMM，在微调与预训练中保持精度，并在 RTX4090 上实现 1.57 倍加速。
摘要：
Transformer 模型已在各类 AI 应用中取得显著成功，但面临高昂的训练成本。INT8 训练等低位训练能够利用吞吐量更高的计算单元，并且已通过块级量化在 GPT2 模型上证明了有效性。然而，它难以适配包含 GLU 单元的现代 Transformer 变体，因为这些变体呈现出复杂的激活异常值分布。为应对该挑战，我们提出 Fallback Quantization（回退量化），实现混合精度 GEMM，对包含异常值的激活块动态地从 8 位回退到 16 位。实验表明，该方法在微调和预训练设置下均表现稳健；此外，该方法在 RTX4090 GPU 上实现了 1.57 倍的端到端训练加速。
英文摘要：
Transformer models have achieved remarkable success across various AI applications but face significant training costs. Low-bit training, such as INT8 training, can leverage computational units with higher throughput, and has already demonstrated its effectiveness on GPT2 models with block-level quantization. However, it struggles with modern Transformer variants incorporating GLU units. This is because those variants demonstrate complex distributions of activation outliers. To address the challenge, we propose Fallback Quantization, implementing mixed-precision GEMM that dynamically falls back 8-bit to 16-bit for activation blocks containing outliers. Experiments show that our approach is robustly competent in both fine-tuning and pretraining settings. Moreover, our method achieves a 1.57x end-to-end training speedup on RTX4090 GPUs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
