arXiv周末暂无论文更新，休息一下吧，周末愉快~~

arXiv 2402.167752024-02cs.CLcs.AI

收藏

大型语言模型量化策略的综合评估

A Comprehensive Evaluation of Quantization Strategies for Large Language Models

College of Intelligence and Computing, Tianjin University（天津大学智能与计算学部）；Xiaomi AI Lab（小米AI实验室）

Renren Jin, Jiangcun Du, Wuwei Huang, Wei Liu, Jian Luan, Bin Wang, Deyi Xiong

备注：: ACL 2024 Findings

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出涵盖知识能力、对齐和效率的三维评估框架，系统评估LLMs量化策略，发现4-bit量化可保持性能且困惑度可作为代理指标，但量化可能降低推理速度。

摘要：

增加大型语言模型（LLMs）的参数数量通常会提升下游任务性能，但也会提高计算和内存成本，使得在资源受限环境中部署变得困难。量化技术通过减少模型权重或激活所需的比特数，以最小的性能损失降低资源需求，并因LLMs的兴起而变得流行。然而，大多数量化研究使用预训练LLMs，量化对指令微调LLMs的影响以及量化LLMs的困惑度与基准性能之间的关系尚不明确。对量化LLMs的评估通常仅限于语言建模和少数分类任务，导致其在其他基准上的表现不清楚。为弥补这些不足，我们提出了一个结构化评估框架，包含三个关键维度：（1）知识与能力，（2）对齐，（3）效率，并在十个多样化基准上进行了广泛实验。实验结果表明，4-bit量化的LLMs能够保持与其非量化对应模型相当的性能，且困惑度在大多数基准上可以作为量化LLMs的代理指标。此外，参数量更大的量化LLMs可以优于较小的LLMs。尽管量化带来了内存节省，但它也可能降低LLMs的推理速度。因此，在量化LLMs的背景下，要实现解码速度和内存消耗的平衡优化，需要大量的工程努力和硬件支持。

英文摘要：

Increasing the number of parameters in large language models (LLMs) usually improves performance in downstream tasks but raises compute and memory costs, making deployment difficult in resource-limited settings. Quantization techniques, which reduce the bits needed for model weights or activations with minimal performance loss, have become popular due to the rise of LLMs. However, most quantization studies use pre-trained LLMs, and the impact of quantization on instruction-tuned LLMs and the relationship between perplexity and benchmark performance of quantized LLMs are not well understood. Evaluation of quantized LLMs is often limited to language modeling and a few classification tasks, leaving their performance on other benchmarks unclear. To address these gaps, we propose a structured evaluation framework consisting of three critical dimensions: (1) knowledge \& capacity, (2) alignment, and (3) efficiency, and conduct extensive experiments across ten diverse benchmarks. Our experimental results indicate that LLMs with 4-bit quantization can retain performance comparable to their non-quantized counterparts, and perplexity can serve as a proxy metric for quantized LLMs on most benchmarks. Furthermore, quantized LLMs with larger parameter scales can outperform smaller LLMs. Despite the memory savings achieved through quantization, it can also slow down the inference speed of LLMs. Consequently, substantial engineering efforts and hardware support are imperative to achieve a balanced optimization of decoding speed and memory consumption in the context of quantized LLMs.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑