超越FLOPs：基于GEMM中心分类法的LLM剪枝真实推理加速基准测试
Beyond FLOPs: Benchmarking Real Inference Acceleration of LLM Pruning under a GEMM-Centric Taxonomy
AI总结：
提出一种基于GEMM维度的剪枝方法分类法，通过统一基准框架系统评估不同剪枝方法在加速-质量帕累托前沿上的表现，发现静态深度剪枝在低质量损失下最优，为LLM剪枝加速提供统一视角。
摘要：
剪枝已成为加速大语言模型（LLM）推理的主流范式，涵盖了一系列方法，这些方法在token、层、头、维度和注意力模式上移除计算。尽管目标相同，这些剪枝方法会引发根本不同的执行行为，导致实际加速效果严重依赖于硬件和内核实现。因此，不同剪枝家族的实际加速收益仍知之甚少。在这项工作中，我们引入了一种基于GEMM中心的分类法，根据通用矩阵乘法（GEMM）的逻辑\textbf{M}、\textbf{N}和\textbf{K}维度重新组织现有剪枝方法。利用这一抽象，我们构建了一个统一的基准测试框架，能够在剪枝设计空间中进行实现一致的比较，并系统地表征加速-质量帕累托前沿。我们的结果表明，静态深度剪枝仍然是最强的帕累托最优基线，并且在内存受限场景下最接近其理论加速上限。在预填充阶段，前沿从低质量损失（0\%--4\%）的静态深度，过渡到中等损失（5\%--16\%）的动态深度，最后到更高损失水平（17\%--26\%）的静态宽度剪枝。这些发现首次建立了基于剪枝的LLM加速实际极限的统一视图，并为未来的剪枝研究提供了指导。\footnote{代码可在 https://github.com/EIT-NLP/LLM-Pruning/tree/main/PruningInferSim 获取。}
英文摘要：
Pruning has emerged as a dominant paradigm for accelerating large language model (LLM) inference, spanning a broad spectrum of methods that remove computation across tokens, layers, heads, dimensions, and attention patterns. Despite sharing the same objective, these pruning approaches induce fundamentally different execution behaviors, causing realized speedups to depend heavily on hardware and kernel implementations. Consequently, the practical acceleration benefits of different pruning families remain poorly understood. In this work, we introduce a GEMM-centric taxonomy that reorganizes existing pruning methods according to the logical \textbf{M}, \textbf{N}, and \textbf{K} dimensions of general matrix multiplication (GEMM). Leveraging this abstraction, we build a unified benchmarking framework that enables implementation-consistent comparison across the pruning design space and systematically characterizes the acceleration--quality Pareto frontier. Our results on Llama3.1-8B show that static depth pruning remains the strongest Pareto-optimal baseline and stays closest to its theoretical acceleration upper bound in memory-bounded scenarios. During prefill, the frontier transitions from static depth at low quality loss (0\%--4\%), to dynamic depth at moderate loss (5\%--16\%), and finally to static width pruning at higher loss levels (17\%--26\%). These findings establish the first unified view of the practical limits of pruning-based LLM acceleration and provide guidance for future pruning research. Code is available at https://github.com/EIT-NLP/LLM-Pruning/tree/main/PruningInferSim
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
