arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.10054cs.LGcs.AI
收藏

通过部分线性前馈网络加速大语言模型

Accelerating Large Language Models through Partially Linear Feed-Forward Network

Institute of Parallel and Distributed Systems, SEIEE, Shanghai Jiao Tong University(上海交通大学电子信息与电气工程学院并行与分布式系统研究所)

Gansen Hu, Zhaoguo Wang, Jinglin Wei, Wei Huang, Haibo Chen

AI总结:

受编译器常量折叠启发提出TARDIS框架,通过部分线性化非线性激活函数实现LLM前馈网络80%参数压缩,在保持高精度的同时带来最高1.6倍推理加速。

摘要:

大语言模型(LLMs)展现出卓越的能力,但由于其庞大的参数量面临部署挑战。虽然现有的剪枝等压缩技术可以减小模型大小,但在高压缩率下会导致显著的准确率下降。我们提出了一种受编译器优化中常量折叠启发的新视角。我们的方法通过将LLMs中的激活函数视为线性函数来实现参数减少。然而,近期的LLMs使用GELU等复杂非线性激活函数,阻碍了该技术的直接应用。我们提出了TARDIS,通过在频繁出现的输入范围内用线性函数部分近似非线性激活函数,实现对LLMs的优化。对于异常值输入,TARDIS采用在线预测器动态回退到原始计算。实验表明,TARDIS在前馈网络中实现了80%的参数减少,同时显著优于最先进的剪枝方法Wanda和RIA,准确率最高提升65%。在7B模型的实际部署中,TARDIS与vLLM服务系统集成时实现了1.6倍端到端推理加速,与广泛采用的HuggingFace实现集成时实现了1.4倍加速,而准确率折损仅为10.9%。

英文摘要:

Large language models (LLMs) demonstrate remarkable capabilities but face deployment challenges due to their massive parameter counts. While existing compression techniques like pruning can reduce model size, it leads to significant accuracy degradation under high compression ratios. We present a novel perspective inspired by constant folding in compiler optimization. Our approach enables parameter reduction by treating activation functions in LLMs as linear functions. However, recent LLMs use complex non-linear activations like GELU that prevent direct application of this technique. We propose TARDIS, which enables optimization of LLMs with non-linear activations by partially approximating them with linear functions in frequently occurring input ranges. For outlier inputs, TARDIS employs an online predictor to dynamically fall back to original computations. Our experiments demonstrate that TARDIS achieves 80% parameter reduction in feed-forward networks, while significantly outperforming state-of-the-art pruning methods Wanda and RIA with up to 65% higher accuracy. In practical deployments for a 7B model, TARDIS achieves 1.6x end-to-end inference speedup when integrated with the vLLM serving system, and 1.4x speedup with the widely adopted HuggingFace implementation, while incurring only a 10.9% accuracy trade-off.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑