正则激活集中度:跨扩散模型架构的列级输出稀疏性特征分析
Activation Concentration: Characterizing Column-Level Output Sparsity Across Diffusion Model Architectures
AI总结:
本文首次系统表征了七种扩散工作负载的列级稀疏性,发现元素级稀疏性高估了硬件可利用稀疏性最多78个百分点,并揭示了UNet+transformer、纯transformer和运动/舞蹈变换器三类工作负载的稀疏性差异及对加速器性能的影响。
摘要:
最近的扩散加速器通过跳过接近零的GELU输出来利用激活稀疏性,报告了52-85%的元素级稀疏性。然而,脉动阵列硬件以列粒度处理激活,单个非零元素迫使整个列被计算。我们首次对跨越三个工作负载组和四种模态的七种扩散工作负载进行了系统的列级稀疏性特征分析。我们的测量表明,元素级稀疏性高估了硬件可利用稀疏性最多78个百分点,并揭示了一个三元分类法。UNet+transformer工作负载表现出激活集中度,具有依赖于工作负载的周期减少高达30.6%。纯transformer DiT显示出分散性,减少12.4%。运动/舞蹈变换器工作负载范围从适度减少到MLD的50.8%,这是由其极端的令牌维度和扩展比驱动的。基于GDDR6的加速器上的周期级模拟证实,内存停顿占总周期的84-89%,并且布局敏感性跟踪基于性能分析的分类法。在五个阈值上的完整精度扫描显示,UNet+transformer工作负载优雅地退化,而运动模型在主要工作点和下一个阈值之间表现出精度悬崖。我们的特征分析表明,工作负载组和模型维度共同决定了列级内存布局优化是否有益,而单独的元素级稀疏性不足以进行该预测。
英文摘要:
Recent diffusion accelerators exploit activation sparsity by skipping near-zero GELU outputs, reporting 52--85% element-level sparsity. However, systolic-array hardware processes activations at column granularity, where a single non-zero element forces the entire column to be computed. We present the first systematic column-level sparsity characterization across seven diffusion workloads spanning three workload groups and four modalities. Our measurements reveal that element-level sparsity overstates hardware-exploitable sparsity by up to 78 percentage points and exposes a three-way taxonomy. UNet+transformer workloads exhibit activation concentration with workload-dependent cycle reductions up to 30.6%. Pure-transformer DiT shows dispersion, yielding 12.4%. Motion/dance transformer workloads range from modest reductions to 50.8% for MLD, driven by its extreme token dimension and expansion ratio. Cycle-level simulation on a GDDR6-based accelerator confirms that memory stalls account for up to 84--89% of total cycles and that layout sensitivity tracks the profiling-based taxonomy. A full accuracy sweep across five thresholds reveals that UNet+transformer workloads degrade gracefully, while motion models exhibit an accuracy cliff between the primary operating point and the next threshold. Our characterization shows that workload group and model dimensions jointly determine whether column-level memory layout optimization is beneficial, and element-level sparsity alone is insufficient for that prediction.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。