ModuLoRA：通过与模块化量化器集成在消费级GPU上微调2位大语言模型
ModuLoRA: Finetuning 2-Bit LLMs on Consumer GPUs by Integrating with Modular Quantizers
AI总结：
本研究提出模块化低秩适配方法ModuLoRA，集成任意量化器与LoRA微调，首次实现2/3位65B参数LLM在单块24GB消费级GPU上的高效微调，性能超越现有4/8位方法，并配套发布LLMTune库。
摘要：
我们提出了一种内存高效的大语言模型（LLM）微调算法，支持仅用一块24GB GPU对650亿参数的LLM进行2/3/4位精度微调。我们的方法——模块化低秩适配（ModuLoRA），将任意用户指定的权重量化器与通过低秩适配器（LoRA）实现的微调相集成。该方法依赖一种简单的与量化无关的反向传播过程，可从定制的黑盒量化模块中自适应生成低精度LLM权重。\n\n这种方法首次实现了2位和3位LLM的微调——利用了最先进的2位QuIP#量化和3位OPTQ量化，性能优于依赖较简单的4位和8位方法的微调。在实验中，ModuLoRA在文本分类、自然语言推理和指令遵循任务上取得了具有竞争力的性能，且内存使用量远低于现有方法；我们还在一个主流摘要任务上超越了最先进的ROUGE分数。我们将ModuLoRA连同一系列低精度模型一起发布，作为LLMTune的一部分——这是一个用于在消费级GPU上量化、运行和微调LLM的用户友好库。
英文摘要：
We propose a memory-efficient finetuning algorithm for large language models (LLMs) that supports finetuning LLMs with 65B parameters in 2/3/4-bit precision on as little as one 24GB GPU. Our method, modular low-rank adaptation (ModuLoRA), integrates any user-specified weight quantizer with finetuning via low-rank adapters (LoRAs). Our approach relies on a simple quantization-agnostic backward pass that adaptively materializes low-precision LLM weights from a custom black-box quantization module. This approach enables finetuning 2-bit and 3-bit LLMs for the first time -- leveraging state-of-the-art 2-bit QuIP\# quantization and 3-bit OPTQ quantization -- outperforming finetuning that relies on less sophisticated 4-bit and 8-bit methods. In our experiments, \lplora~attains competitive performance on text classification, natural language inference, and instruction following tasks using significantly less memory than existing approaches, and we also surpass the state-of-the-art ROUGE score on a popular summarization task. We release \lplora~together with a series of low-precision models as part of \llmtune, a user-friendly library for quantizing, running, and finetuning LLMs on consumer GPUs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
