MXFormer: 一种微缩浮点电荷陷阱晶体管计算在内存变换器加速器
MXFormer: A Microscaling Floating-Point Charge-Trap Transistor Compute-in-Memory Transformer Accelerator
AI总结：
MXFormer通过微缩浮点电荷陷阱晶体管实现高计算密度和能效的变换器加速器，支持大规模短序列模型的高效推理。
摘要：
变换器模型的普及往往受到部署时显著的计算和内存带宽需求的限制。为了解决这个问题，我们提出了MXFormer，一种新型的混合、权重静态计算在内存（CIM）加速器，为大型短序列变换器的固定模型推断提供高吞吐量和效率。我们的架构基础是使用超密集电荷陷阱晶体管（CTTs）在微缩MXFP4 CIM阵列中，唯一地使芯片上能够以全权重静态（FWS）方式存储多达数千万个参数。我们引入了一种静态划分的设计，有12个变换器块通过深度流水的数据流连接。静态权重层（MLPs和线性投影）在高度并行的模拟CTT阵列上执行，使用MXFP4原生流程，具有每块指数对齐和10位SAR ADC。动态计算在完全准确的数字块中处理，利用MXFP启用的脉冲阵列进行缩放点积注意力和向量单元进行层归一化和FlashAttention式Softmax。通过消除所有权重移动，深度流水的MXFormer架构实现了非常高的单流吞吐量和效率，在ViT-L/32（双芯片）上处理58275 FPS，在ViT-B/16（单芯片）上处理41269 FPS。MXFormer在计算密度方面比同等先进的非FWS数字、混合和光子变换器加速器高出约3.3倍至60.5倍，在能耗效率方面高出约1.7倍至2.5倍。与FWS加速器相比，MXFormer在计算密度上提高了约20.9倍，在驻留权重存储密度上提高了约2倍，同时保持接近数字精度（精度下降低于1%）而无需任何模型重新训练。
英文摘要：
The proliferation of Transformer models is often constrained by the significant computational and memory bandwidth demands of deployment. To address this, we present MXFormer, a novel, hybrid, weight-stationary Compute-in-Memory (CIM) accelerator that provides high throughput and efficiency for fixed-model inference on large short-sequence Transformers. Our architecture's foundation is the use of ultra-dense Charge-Trap Transistors (CTTs) in Microscaling MXFP4 CIM arrays, uniquely enabling the on-chip storage of up to hundreds of millions of parameters in Fully Weight Stationary (FWS) fashion. We introduce a statically partitioned design with 12 Transformer blocks connected by a deeply pipelined dataflow. Static-weight layers (MLPs and linear projections) execute on highly parallel analog CTT arrays using an MXFP4-native flow with per-block exponent alignment and a 10-bit SAR ADC. Dynamic computations are handled in fully accurate digital blocks that utilize MXFP-enabled systolic arrays for scaled dot-product attention and vector units for LayerNorm and FlashAttention-style Softmax. By eliminating all weight movement, the deeply pipelined MXFormer architecture yields very high single-stream throughput and efficiency, processing 58275 FPS on ViT-L/32 (dual-chip) or 41269 FPS on ViT-B/16 (single chip). MXFormer outperforms comparable state-of-the-art non-FWS digital, hybrid and photonic Transformer accelerators ~3.3x-60.5x in compute density and ~1.7x-2.5x in energy efficiency. Against FWS accelerators, MXFormer improves compute density by ~20.9x and resident weight storage density by ~2x, while preserving near-digital accuracy (drop of <1%) without any model retraining.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
