arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.01803cs.LGcs.CL
收藏

STBLLM:通过结构化二值化大语言模型突破1比特壁垒

STBLLM: Breaking the 1-Bit Barrier with Structured Binary LLMs

HKUST(GZ)(香港科技大学(广州));HKUST(香港科技大学);SYSU(中山大学);HKBU(香港浸会大学);HIT(SZ)(哈尔滨工业大学(深圳))

Peijie Dong, Lujun Li, Yuedong Zhong, Dayou Du, Ruibo Fan, Yuhan Chen, Zhenheng Tang, Qiang Wang, Wei Xue, Yike Guo, Xiaowen Chu

AI总结:

针对大语言模型推理内存受限问题,提出STBLLM结构化二值化方法,通过标准化重要性指标、分层稀疏化等技术将模型压缩至1比特以下,在多个主流模型上实现内存大幅降低且性能优于同类二值化方法。

摘要:

本文提出了首个将大语言模型(LLM)压缩至1比特以下精度的结构化二值化方法。尽管LLM已取得显著性能,但其推理阶段的内存受限特性阻碍了在资源受限设备上的部署。通过二值化将权重降至1比特精度可大幅提升计算效率。我们观察到,二值化LLM中的部分权重可被随机翻转而不会造成明显性能下降,这表明存在进一步压缩的潜力。为利用这一特性,STBLLM采用N:M稀疏化技术实现权重的结构化二值化。具体而言,我们提出了一种新颖的标准化重要性(SI)指标,该指标结合权重量级和输入特征范数以更准确地评估权重重要性。随后,我们提出分层方法,允许LLM的不同层以不同的N:M比例进行稀疏化,从而平衡压缩率与精度。此外,我们对较不重要的权重实施细粒度分组策略,对稀疏区、中间区和稠密区应用不同的量化方案。最后,我们设计了专用CUDA内核以支持结构化二值化。我们在LLaMA-1/2/3、OPT系列和Mistral上开展了大量实验以评估STBLLM的有效性。结果表明,该方法在显著降低内存需求的同时,性能优于其他压缩二值化LLM方法。

英文摘要:

In this paper, we present the first structural binarization method for LLM compression to less than 1-bit precision. Although LLMs have achieved remarkable performance, their memory-bound nature during the inference stage hinders the adoption of resource-constrained devices. Reducing weights to 1-bit precision through binarization substantially enhances computational efficiency. We observe that some weights in binarized LLMs can be randomly flipped without significant performance degradation, suggesting the potential for further compression. To exploit this, our STBLLM employs an N:M sparsity technique to achieve structural binarization of the weights. Specifically, we introduce a novel Standardized Importance (SI) metric, which considers weight magnitude and input feature norm to more accurately assess weight significance. Then, we propose a layer-wise approach, allowing different layers of the LLM to be sparsified with varying N:M ratios, thereby balancing compression and accuracy. Furthermore, we implement a fine-grained grouping strategy for less important weights, applying distinct quantization schemes to sparse, intermediate, and dense regions. Finally, we design a specialized CUDA kernel to support structural binarization. We conduct extensive experiments on LLaMA-1/2/3, OPT family, and Mistral to evaluate the effectiveness of STBLLM. The results demonstrate that our approach performs better than other compressed binarization LLM methods while significantly reducing memory requirements.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑