为何更小更慢?压缩大语言模型中的维度不匹配
Why Smaller Is Slower? Dimensional Misalignment in Compressed LLMs
AI总结:
本文研究压缩大语言模型后出现的维度不匹配问题,提出GAC压缩方法,通过多选背包优化实现硬件对齐,提升推理速度1.5倍。
摘要:
后训练压缩减少了LLM参数数量,但通常会产生不规则张量维度,从而降低GPU性能——我们称之为维度不匹配。我们进行了全栈分析,追溯框架、库和硬件三个层面的根源。关键发现是,模型推理变慢是因为生成的维度与GPU执行栈不匹配。例如,使用激活感知奇异值分解(ASVD)压缩Llama-3-8B,参数减少15%,但速度并未加快,因为95%的维度不匹配。我们提出GAC(GPU-Aligned Compression),一种新的压缩范式,通过多选背包优化重新选择硬件对齐的维度,实现100%对齐并恢复1.5倍的速度提升,同时保持模型质量。
英文摘要:
Post-training compression reduces LLM parameter counts but often produces irregular tensor dimensions that degrade GPU performance -- a phenomenon we call \emph{dimensional misalignment}. We present a full-stack analysis tracing root causes at three levels: framework, library, and hardware. The key insight is that model inference becomes slower because the resulting dimensions are unfriendly with the GPU execution stack. For example, compressing Llama-3-8B with activation-aware singular value decomposition (ASVD) has 15\% fewer parameters yet runs no faster than the uncompressed baseline, because 95\% of its dimensions are misaligned. We propose \textbf{GAC} (GPU-Aligned Compression), a new compression paradigm that wraps any dimension-reducing compressor and re-selects hardware-aligned dimensions via multi-choice knapsack optimization under the same parameter budget. We evaluate GAC on Llama-3-8B with ASVD and LLM-Pruner, achieving 100\% alignment and recovering up to 1.5$\times$ speedup while preserving model quality.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。