arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.09748cs.CLcs.AIcs.LGcs.PF
收藏

大语言模型的模型压缩与高效推理:综述

Model Compression and Efficient Inference for Large Language Models: A Survey

School of Software Technology, Zhejiang University(浙江大学软件学院);School of Mathematical Sciences, Zhejiang University(浙江大学数学科学学院)

Wenxiao Wang, Wei Chen, Yicong Luo, Yongliu Long, Zhengkai Lin, Liye Zhang, Binbin Lin, Deng Cai, Xiaofei He

AI总结:

本综述从算法角度梳理了大语言模型的压缩与高效推理方法,指出大模型在免调优压缩和保持通用泛化能力上的两大特征,并介绍了支持部署的成熟推理框架。

摘要:

基于Transformer的大语言模型取得了巨大成功。然而,推理过程中产生的显著内存和计算成本使得在资源受限的设备上部署大模型充满挑战。在本文中,我们从算法角度研究了大语言模型的压缩和高效推理方法。在分类法方面,与较小模型类似,大语言模型的压缩和加速算法仍可分为量化、剪枝、蒸馏、紧凑架构设计和动态网络。然而,与较小模型相比,大语言模型有两个突出特征:(1)大多数压缩算法在压缩后需要微调甚至重新训练模型。大模型最显著的方面是与模型微调或训练相关的高昂成本。因此,许多针对大模型的算法(如量化和剪枝)开始探索免调优算法。(2)大模型强调通用性和泛化能力,而非单一任务上的性能。因此,许多算法(如知识蒸馏)关注如何在压缩后保持其通用性和泛化能力。由于这两个特征在早期大模型中并不十分明显,我们进一步将大语言模型区分为中等模型和“真正”的大模型。此外,我们还介绍了一些用于大模型高效推理的成熟框架,这些框架能够支持基础的压缩或加速算法,极大地便利了用户的模型部署。

英文摘要:

Transformer based large language models have achieved tremendous success. However, the significant memory and computational costs incurred during the inference process make it challenging to deploy large models on resource-constrained devices. In this paper, we investigate compression and efficient inference methods for large language models from an algorithmic perspective. Regarding taxonomy, similar to smaller models, compression and acceleration algorithms for large language models can still be categorized into quantization, pruning, distillation, compact architecture design, dynamic networks. However, Large language models have two prominent characteristics compared to smaller models: (1) Most of compression algorithms require finetuning or even retraining the model after compression. The most notable aspect of large models is the very high cost associated with model finetuning or training. Therefore, many algorithms for large models, such as quantization and pruning, start to explore tuning-free algorithms. (2) Large models emphasize versatility and generalization rather than performance on a single task. Hence, many algorithms, such as knowledge distillation, focus on how to preserving their versatility and generalization after compression. Since these two characteristics were not very pronounced in early large models, we further distinguish large language models into medium models and ``real'' large models. Additionally, we also provide an introduction to some mature frameworks for efficient inference of large models, which can support basic compression or acceleration algorithms, greatly facilitating model deployment for users.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑