arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.09595cs.DCcs.AI
收藏

为何更小更慢?压缩大语言模型中的维度不匹配

Why Smaller Is Slower? Dimensional Misalignment in Compressed LLMs

KAUST(阿卜杜拉国王科技大学);HUMAIN

Jihao Xin, Tian Lyu, Qilong Pan, Kesen Wang, Marco Canini

AI总结:

本文研究压缩大语言模型后出现的维度不匹配问题,提出GAC压缩方法,通过多选背包优化实现硬件对齐,提升推理速度1.5倍。

摘要:

后训练压缩减少了LLM参数数量,但通常会产生不规则张量维度,从而降低GPU性能——我们称之为维度不匹配。我们进行了全栈分析,追溯框架、库和硬件三个层面的根源。关键发现是,模型推理变慢是因为生成的维度与GPU执行栈不匹配。例如,使用激活感知奇异值分解(ASVD)压缩Llama-3-8B,参数减少15%,但速度并未加快,因为95%的维度不匹配。我们提出GAC(GPU-Aligned Compression),一种新的压缩范式,通过多选背包优化重新选择硬件对齐的维度,实现100%对齐并恢复1.5倍的速度提升,同时保持模型质量。

英文摘要:

Post-training compression reduces LLM parameter counts but often produces irregular tensor dimensions that degrade GPU performance -- a phenomenon we call \emph{dimensional misalignment}. We present a full-stack analysis tracing root causes at three levels: framework, library, and hardware. The key insight is that model inference becomes slower because the resulting dimensions are unfriendly with the GPU execution stack. For example, compressing Llama-3-8B with activation-aware singular value decomposition (ASVD) has 15\% fewer parameters yet runs no faster than the uncompressed baseline, because 95\% of its dimensions are misaligned. We propose \textbf{GAC} (GPU-Aligned Compression), a new compression paradigm that wraps any dimension-reducing compressor and re-selects hardware-aligned dimensions via multi-choice knapsack optimization under the same parameter budget. We evaluate GAC on Llama-3-8B with ASVD and LLM-Pruner, achieving 100\% alignment and recovering up to 1.5$\times$ speedup while preserving model quality.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑